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Preface 


The  RAND  Corporation  has  worked  with  the  Army  Medical 
Department  (AMEDD)  on  a  project  entitled  “Implementing  Clinical 
Practice  Guidelines  in  the  Army  Medical  System.”  This  project  was 
undertaken  to  assist  the  AMEDD  in  developing  and  testing  methods 
to  effectively  implement  clinical  practice  guidelines  in  Army  military 
treatment  facilities  (MTEs),  with  the  goal  being  to  achieve  consistent 
and  quality  clinical  practices  across  the  Army  health  system.  Three 
demonstrations  were  conducted  to  test  and  refine  methods  before 
embarking  on  full  guideline  implementation  across  the  Army  health 
system.  These  demonstrations  tested  use  of  guidelines  for  primary 
care  management  of  low  back  pain,  asthma,  and  diabetes. 

This  report  presents  the  final  findings  from  the  RAND  evalua¬ 
tion  of  the  diabetes  practice  guideline  demonstration,  which  was 
conducted  in  2000  and  2001.  The  evaluation  included  both  an 
assessment  of  the  implementation  process  and  a  quantitative  analysis 
of  changes  in  clinical  practices.  The  quantitative  analysis  was  per¬ 
formed  to  document  the  extent  to  which  intended  actions  were  actu¬ 
ally  implemented  by  the  MTEs,  assess  short-term  effects  on  clinical 
practices,  develop  and  test  metrics  and  measurement  methods  that 
can  be  adopted  by  the  AMEDD  for  routine  monitoring  of  progress, 
and  assess  the  quality  and  limitations  of  available  data  for  monitoring 
practice  improvements  and  clinical  outcomes.  Recommendations  for 
future  actions  by  the  AMEDD  are  presented. 

This  report  is  one  of  three  final  reports  being  generated  in  this 
project.  It  should  be  of  interest  to  anyone  concerned  with  military 
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medical  systems  and  policies.  Similar  reports  were  prepared  from  the 
demonstrations  for  the  low  hack  pain  and  asthma  practice  guidelines. 

This  research  was  sponsored  hy  the  U.S.  Army  Surgeon  General. 
It  was  conducted  jointly  hy  the  RAND  Arroyo  Center,  a  federally 
funded  research  and  development  center  sponsored  hy  the  U.S. 
Army,  and  hy  the  RAND  Center  for  Military  Health  Policy  Research. 
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Summary 


Background 

The  Department  of  Defense  (DoD)  and  Department  of  Veterans 
Affairs  (VA)  initiated  a  collaborative  project  in  early  1998  to  establish 
a  single  standard  of  care  in  the  military  and  VA  health  systems.  This 
initiative  established  evidence-based  clinical  practice  guidelines  for 
selected  conditions  that  would  be  applied  in  all  DoD  and  VA  health 
facilities.  Each  practice  guideline  is  a  statement  of  best  practices  for 
the  management  and  treatment  of  the  health  condition  it  addresses. 
For  each  guideline,  the  DoD/VA  Working  Group  designated  an 
expert  panel  to  develop  the  guideline  contents  and  relevant  metrics, 
which  were  made  available  for  use  by  the  military  services  and  VA 
health-care  facilities. 

The  Army  Medical  Department  (AMEDD)  has  made  a  com¬ 
mitment  to  establishing  a  structure  and  process  to  support  its  military 
treatment  facilities  (MTFs)  in  implementing  evidence-based  practice 
guidelines  with  the  goal  of  achieving  best  practices  that  reduce  varia¬ 
tion  and  enhance  quality  of  medical  care.  The  AMEDD  contracted 
with  RAND  to  work  as  a  partner  in  the  development  and  testing  of 
guideline  implementation  methods  for  ultimate  application  to  an 
Army-wide  guideline  program. 

Taking  the  approach  of  testing  new  methods  on  a  small  scale, 
AMEDD  fielded  three  demonstrations  over  a  two-year  period,  each 
testing  different  clinical  practice  guidelines.  All  three  of  the  practice 
guidelines — for  lower  back  pain,  asthma,  and  diabetes — were  estab¬ 
lished  collaboratively  by  the  VA  and  DoD.  This  report  presents  the 
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results  from  a  RAND  evaluation  of  the  diabetes  practice  guideline 
demonstration.  The  principal  emphasis  of  the  practice  guideline  for 
primary  care  management  of  diabetes  was  on  effective  management  of 
blood-sugar  levels  with  the  goal  of  preventing  short-term  complica¬ 
tions  and  long-term  effects  on  organ  systems. 

The  key  elements  of  the  Diabetes  Practice  Guideline  were  the 
following: 

•  patient  evaluation, 

•  achieving  and  maintaining  glycemic  control, 

•  patient  education  and  counseling,  and 

•  early  detection  and  management  of  diabetic  complications. 


Approach 

AMEDD  began  the  demonstration  process  in  AMEDD’s  Western 
Region  with  a  kickoff  meeting  in  December  1999  (implementation 
processes  and  tools  are  summarized  in  the  list  below).  Two  MTEs 
participated  in  the  demonstration  as  designed:  Madigan  Army  Medi¬ 
cal  Center  (AMC),  Eort  Eewis,  Washington,  a  large,  urban  specialty 
medical  center,  and  Bassett  Army  Community  Hospital  (ACH),  Eort 
Wainwright,  Alaska,  a  small  hospital  at  a  remote  outpost.  Three  other 
Army  MTEs  concurrently  implemented  the  diabetes  guideline  in  a 
separate  demonstration.'  Data  for  all  five  MTEs  were  included  in  the 
evaluation  of  guideline  effects. 

The  following  processes  and  tools  were  used  in  guideline 
implementation: 

•  the  practice  guideline  and  metrics. 


'  In  this  three-year  demonstration  of  a  program  called  TRICARE  Senior  Prime,  DoD  con¬ 
tracted  with  Medicare  to  offer  Medicare  managed-care  plans  in  six  locations  for  DoD  benefi¬ 
ciaries  who  also  were  Medicare-eligible.  One  participation  requirement  for  the  Senior  Prime 
plans  was  to  implement  a  quality  improvement  initiative;  diabetes  care  was  chosen.  The  three 
sites  for  which  data  were  included  in  this  report  were  Brooke  AMC  at  Fort  Sam  Houston, 
Texas,  Evans  ACH  at  Fort  Carson,  Colorado,  and  Reynolds  ACH  at  Fort  Sill,  Oklahoma. 
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•  a  guideline  toolkit  of  materials  to  support  implementation 
activities, 

•  a  kickoff  conference  to  develop  implementation  action  plans, 

•  MTF  implementation  activities  to  carry  out  the  action  plans, 
information  exchange  between  teams  and  with  MEDCOM  to 
share  experiences  and  huild  on  successes, 

•  ongoing  support  of  MEDCOM  to  include  revision  and  devel¬ 
opment  of  toolkit  items,  and 

•  monitoring  of  implementation  progress  hy  hoth  MEDCOM 
and  the  participating  MTEs. 

RAND’s  evaluation  included  an  assessment  of  the  implementa¬ 
tion  process,  an  attempt  to  establish  preimplementation  baseline 
measurements  as  benchmarks,  an  assessment  of  the  effects  of  the 
guideline  implementation  on  care  processes  one  year  later,  and  an 
evaluation  of  methods  available  to  and  developed  by  AMEDD  to 
measure  outcomes  at  its  facilities.  The  specific  methods  and  data  used 
in  the  evaluation  are  described  in  Chapter  Two  and  Appendix  A. 

Implementation  Evaluation 

Earlier  demonstrations  had  shown  the  value  of  using  a  systems 
approach,  which  involved  achieving  “buy-in”  from  the  staff  responsi¬ 
ble  for  implementing  the  new  practices  and  ensuring  that  clinical  and 
administrative  systems  are  in  place  to  facilitate  staff  adherence  to  the 
guideline.  The  purposes  of  the  process  evaluation  were  to  document 
the  actions  and  experiences  of  the  participating  MTEs;  identify  areas 
where  AMEDD  policies,  systems,  and  processes  could  be  strength¬ 
ened;  and  assess  the  degree  to  which  AMEDD  can  apply  lessons  from 
the  demonstration  to  implement  the  diabetes  guideline  across  its  sys¬ 
tem.  A  participant-observer  approach  was  used  to  learn  from  the 
MTEs’  experiences,  provide  feedback,  and  facilitate  shared  learning 
among  the  MTEs.  Information  was  collected  from  the  participating 
MTEs  through  two  site  visits  (one  at  four  months  and  one  at  ten 
months),  monthly  progress  reports  prepared  by  the  MTEs,  and  ques¬ 
tionnaires  completed  by  individual  participants. 
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The  Outcomes  Evaluation 

The  purposes  of  analyzing  the  outcomes  of  guideline  implementation 
were  to  document  the  extent  to  which  intended  actions  were  actually 
implemented  hy  the  MTFs,  monitor  short-term  effects  on  service 
delivery  methods  and  activity,  and  develop  and  test  metrics  and 
measurement  methods  that  can  he  adopted  hy  the  MTFs  and 
MEDCOM  for  routine  monitoring  of  progress.  Outcomes  were 
evaluated  using  two  sets  of  indicators.  First,  the  participating  MTFs 
assessed  their  own  compliance  with  a  set  of  indicators  developed  hy 
the  nationwide  Diabetes  Quality  Improvement  Project  (DQIP)  and 
adopted  hy  the  DoD/VA  Diabetes  Working  Group  as  guideline  met¬ 
rics.  RAND  established  a  second  set  of  five  outcome  indicators  that 
could  be  measured  using  administrative  data  from  the  DoD  health 
system.  Of  these  five  indicators,  one — annual  eye  examinations — 
reflected  DQIP  standards.  The  other  four  were  primary  care  visits, 
use  of  oral  hypoglycemic  agents,  emergency  room  (ER)  visits,  and 
inpatient  stays.  Other  DQIP  indicators  could  not  be  assessed  using 
administrative  data.  These  included  foot  exams,  referrals  to  diabetes 
education  services,  and  assessment  for  nephropathy  because  such 
information  was  collected  and  stored  only  at  the  local  MTE  level. 

To  assess  the  effects  of  the  demonstration,  we  used  a  time  series, 
control  comparison  design  to  assess  changes  in  values  of  the  MTEs’ 
performance  indicators  over  time.  While  the  kickoff  meeting  was  held 
in  December  1999,  we  considered  April  1,  2000,  to  be  the  date  when 
the  guideline  might  impact  patient  care  and  thus  defined  the  baseline 
period  as  the  year  preceding  this  intervention  date.  To  control  for 
temporal  trends  that  might  account  for  observed  changes  in  the  indi¬ 
cators,  we  also  compared  the  data  for  the  demonstration  sites  to  those 
of  a  set  of  matched  control  MTEs  that  had  not  implemented  the  dia¬ 
betes  guideline.  These  comparison  MTEs  were  selected  for  similarity 
to  the  demonstration  MTEs.  Eor  the  time  trend  comparisons,  we 
analyzed  one  year  of  baseline  data  for  the  demonstration  sites  (April 
1999  through  March  2000)  and  one  year  of  data  collected  after 
introduction  of  the  guideline  (April  2000  through  March  2001). 

The  patient  sample  used  for  these  analyses  was  a  subset  of  all 
patients  who  were  enrolled  in  TRICARE  Prime  at  one  of  the  five 
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demonstration  MTFs  (the  two  MTFs  in  our  demonstration  plus  the 
three  Senior  Prime  MTFs)  or  five  comparison  MTFs  during  the 
study  period.  For  each  indicator,  we  calculated  averages  for  the  sam¬ 
ple  of  diabetic  patients  continuously  enrolled  at  each  MTF  during  the 
baseline  period  and  an  overall  average  value  for  both  control  and 
demonstration  MTFs. 

To  gain  perspective  on  how  the  demonstration  participants 
reflected  diabetes  patients  served  by  Army  facilities,  we  also  docu¬ 
mented  the  number  and  characteristics  of  all  DoD  beneficiaries  who 
were  identified  as  having  diabetes  and  who  used  an  Army  MTF  at 
any  time  during  the  study  period,  based  on  International  Classifica¬ 
tion  of  Diseases,  Ninth  Revision  (ICD-9),  diagnostic  codes  on  MTF 
encounter  records,  or  network  provider  payment  claims. 


Findings  and  Implications 

Army  medical  facilities  served  close  to  220,000  diabetic  patients 
during  the  first  year  of  our  study  and  more  than  230,000  diabetic 
patients  during  the  second  year,  more  than  half  of  whom  were  per¬ 
sonnel,  retirees,  or  family  members  of  other  (non-Army)  military 
services.  Among  those  affiliated  with  the  Army,  all  but  a  small  frac¬ 
tion  were  either  retired  Army  personnel  or  their  family  members. 
Only  a  small  number  were  active-duty  Army  personnel:  Overall,  42.8 
percent  of  the  diabetic  patients  in  the  first  year  were  45  to  64  years  of 
age,  and  46.2  percent  were  65  years  of  age  or  older.  The  percentages 
were  similar  for  the  second  study  year. 

The  patients  in  our  sample  used  both  MTFs  and  network  pro¬ 
viders  for  their  diabetes  care.  Only  61.8  percent  of  total  diabetes- 
related  visits  to  MTF  outpatient  clinics  or  ERs  were  by  patients 
enrolled  in  TRICARE  Prime  at  the  MTEs.  Another  37.6  percent  of 
these  MTE  visits  were  for  nonenrolled  patients,  and  less  than  1  per¬ 
cent  of  the  visits  were  for  patients  enrolled  with  network  providers. 
By  contrast,  all  but  a  small  percentage  of  diabetes-related  hospital 
inpatient  stays  at  MTEs  were  for  their  own  enrollees.  This  finding  has 
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implications  for  both  patient  management  and  outcome  measure¬ 
ment. 

Baseline  Diabetes  Care  Performance  Measures 

Baseline  values  varied  considerably  for  all  indicators:  average  number 
of  primary  care  visits  per  100  patients,  percentages  of  non-insulin 
dependent  patients  who  were  treated  with  oral  agents,  percentages  of 
diabetes  patients  who  had  at  least  one  eye  examination  during  the 
year,  rates  of  ER  visits,  and  rates  of  inpatient  stays.  No  practice 
guidelines  yet  define  the  optimal  number  of  primary  care  visits  and 
the  use  of  oral  agents  because  appropriate  measures  depend  on  indi¬ 
vidual  patient  needs  and  clinical  judgment.  Nevertheless,  the  wide 
variation  in  practices  among  facilities  suggests  that  under-  or  over¬ 
treatment  may  be  a  concern.  Baseline  levels  of  annual  eye  exams,  an 
indicator  for  which  guidelines  exist,  were  uniformly  low,  suggesting 
the  need  to  investigate  possible  underlying  causes. 

Critical  Factors  for  Implementing  Practice  Improvements 

Drawing  on  published  literature  on  implementation  of  practice 
guidelines  and  the  implementation  experiences  observed  in  the 
AMEDD  lower  back  pain  and  asthma  guideline  demonstrations,  we 
identified  six  factors  that  critically  influence  the  successful  integration 
of  new  practices  into  clinical  and  administrative  processes.  We 
assessed  the  performance  of  the  diabetes  guideline  demonstration 
MTE s  on  these  factors. 

•  Command  leadership  commitment  at  the  MTE,  regional, 
and  corporate  levels.  The  diabetes  implementation  teams  had 
the  support  of  both  the  MTE  commands  as  well  as  the  leader¬ 
ship  of  the  TRICARE  Region  1 1  Eead  Agent  office,  which 
planned  to  implement  this  approach  for  other  MTEs  in  the 
region. 

•  Monitoring  progress.  The  performance  of  the  demonstration 
MTEs  in  the  area  of  monitoring  was  mixed.  Of  the  two  demon¬ 
stration  MTEs  (not  including  the  Senior  Prime  sites),  one 
actively  measured  trends  in  performance  on  the  DQIP  measures. 
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while  the  other  MTF  struggled  to  extract  the  needed  data  in  the 
face  of  inadequate  staffing  levels  and  technical  prohlems  with  its 
data  system.  Data  system  harriers  also  prevented  both  MTFs 
from  establishing  a  local  diabetes  registry. 

•  Guidance  and  support  to  the  MTFs  by  MEDCOM.  By  the 
time  the  diabetes  guideline  demonstration  began,  MEDCOM 
had  well-established  staffing  and  other  resources  and  was  pro¬ 
viding  policy  guidance  and  technical  support  to  help  MTFs 
implement  practice  improvements  for  diabetes  care.  We  believe 
MEDCOM’s  committed  support  has  been  a  strong  foundation 
for  the  practice  improvement  efforts  of  the  demonstrations. 

•  Guideline  champions  who  are  opinion  leaders.  The  partici¬ 
pating  MTFs  identified  well-respected  physicians  to  serve  as 
guideline  champions  for  the  diabetes  demonstration,  and  these 
physicians  showed  a  commitment  to  leading  the  implementa¬ 
tion  activities.  However,  the  champions  were  permitted  to  make 
only  limited  commitments  to  the  initiative. 

•  Resource  support  for  champions.  Although  both  MTF  com¬ 
manders  authorized  the  champions  to  lead  the  implementation 
of  the  diabetes  guideline,  neither  champion  received  tangible 
resource  support  for  the  activities  (other  than  attendance  at  the 
kickoff  conference).  Nevertheless,  facilitators  designated  by  the 
commanders  at  both  MTFs  were  responsible  for  providing  staff 
support  for  the  champions. 

•  Institutionalization  of  new  practices.  The  participating  MTFs 
made  some  progress  toward  achieving  practices  consistent  with 
the  diabetes  guideline,  focusing  on  areas  where  their  perfor¬ 
mance  on  DQIP  measures  was  the  weakest.  To  achieve  sus¬ 
tained  improvements,  they  will  need  to  both  conduct  regular 
education  sessions  for  providers,  clinic  staff,  and  newcomers  to 
the  MTF  and  deliver  regular  feedback  to  providers  on  perfor¬ 
mance  trends  for  the  DQIP  measures. 


Effects  of  the  Demonstration  on  Performance  Measures 

Data  from  both  the  local  MTFs  and  the  centralized  data  system  can 
and  should  be  used  for  monitoring  progress  of  the  MTFs  on  per- 
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formance  indicators  for  diabetes  care  (or  any  other  health  condition) . 
Based  on  process  evaluation  information  and  our  analyses  of  encoun¬ 
ter  data,  we  examined  trends  reported  by  the  demonstration  MTFs 
for  the  DQIP  performance  indicators  they  monitored,  and  we  also 
analyzed  trends  in  diabetes  care  service  utilization  that  we  could 
obtain  from  administrative  data  for  both  demonstration  and  control 
sites. 

MTF  Monitoring  of  DQIP  Indicators.  Four  of  the  five  demonstra¬ 
tion  MTFs  reported  that  they  had  begun  to  collect  data  on  the  DQIP 
measures  using  either  their  clinical  data  systems  or  medical  charts  as 
data  sources.  Three  of  these  MTFs  reported  an  improvement  in  their 
performance  between  baseline  and  12  months  into  the  demonstra¬ 
tion.  Such  improvements  could  lead  to  an  eventual  reduction  in  dia¬ 
betes  complications  and  associated  avoidable  health-care  events  (e.g., 
ER  visits  or  hospitalizations). 

In  our  review  of  the  materials  the  MTFs  provided,  several  issues 
arose  regarding  data  quality  and  comparability  across  MTFs,  includ¬ 
ing  incomplete  or  ambiguous  indicator  definitions  (e.g.,  percentage  of 
patients  receiving  a  lipids  panel  versus  the  percentage  of  patients  with 
LDL  levels  in  the  normal  range) . 

RAND  Analysis  of  Service  Utilization  Trends.  The  performance 
of  the  demonstration  MTFs  on  the  service  delivery  indicators  we 
measured  did  not  change  substantially  between  baseline  and  the  end 
of  the  first  demonstration  year: 

•  Primary  care  visit  rates  held  steady  during  the  first  two  quarters 
of  the  first  demonstration  year  and  then  decreased  in  the  last 
two  quarters. 

•  Use  of  oral  hypoglycemic  agents  at  demonstration  MTFs 
increased  from  baseline  during  the  demonstration  period,  as 
expected,  but  this  increase  did  not  differ  significantly  from  that 
of  the  control  MTFs. 

•  The  percentage  of  patients  with  diabetes-related  annual  eye 
examinations  increased  significantly  at  demonstration  MTFs, 
but  it  was  not  clear  whether  this  was  a  real  increase  or  the  result 
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of  improved  coding  for  the  diabetes  diagnosis  on  the  encounter 
records. 

•  Neither  ER  visit  rates  nor  hospitalization  rates — indicators  of 
potentially  avoidable  health-care  events — changed  during  the 
demonstration. 

RAND  Analysis  of  MTF  Cost  Trends.  The  introduction  of  the 
diabetes  practice  guideline  did  not  appear  to  affect  MTF  costs  in  the 
first  demonstration  year: 

•  As  a  proportion  of  total  costs  of  diabetic  care  per  patient  and  per 
MTF,  costs  of  care  for  nonenrollees  was  substantial  at  both 
demonstration  and  control  hospitals.  Nonenrollee  inpatient 
costs  far  exceeded  enrollee  inpatient  costs.  Many  of  the  nonen¬ 
rollees  were  over-65  Medicare  recipients. 

•  For  enrollees,  per-patient  costs  at  demonstration  hospitals 
exceeded  those  of  control  hospitals  for  both  inpatient  and  out¬ 
patient  care  and  in  both  study  years.  From  year  one  to  year  two, 
average  per-patient  costs  for  both  outpatient  and  inpatient  care 
increased  slightly  at  the  demonstration  sites,  while  at  the  control 
sites,  outpatient  costs  rose  slightly  and  inpatient  costs  fell. 

•  For  nonenrollees,  per-patient  costs  at  demonstration  hospitals 
were  comparable  to  or  slightly  less  than  those  of  control  hospi¬ 
tals  for  both  study  years.  From  year  one  to  year  two,  average  per- 
patient  costs  for  both  outpatient  and  inpatient  care  fell  slightly 
at  demonstration  sites,  while  at  the  control  sites,  outpatient  and 
inpatient  costs  rose  slightly. 

•  From  one  MTF  to  another,  per-patient  costs  for  both  outpatient 
and  inpatient  services  varied  widely. 


The  contrast  between  improvements  on  the  DQIP  indicators 
reported  by  the  demonstration  MTFs  and  the  virtual  absence  of 
changes  in  the  cost  indicators  we  analyzed  suggests  that  our  measures 
did  not  capture  the  full  dynamics  of  the  process  changes  made  by  the 
MTFs  to  achieve  their  reported  improvements  on  the  DQIP  indica- 
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tors,  while  administrative  data  can  be  used  to  count  events  (e.g.,  vis¬ 
its),  they  cannot  be  used  to  assess  the  contents  of  those  events  (e.g., 
diabetes  education,  foot  exams,  or  referrals).  Although  we  were 
familiar  with  the  action  strategies  of  the  two  MTFs  in  the  AMEDD 
demonstration  and  the  specific  processes  they  were  attempting  to 
modify,  we  could  not  develop  indicators  that  measured  those  changes 
using  administrative  data,  with  the  exception  of  annual  eye  exams. 

Other  possible  contributors  to  the  apparently  limited  effects  of 
the  demonstration  include  the  following: 

•  the  time  between  implementation  and  measurement  may  have 
been  too  short  for  the  guideline  to  have  affected  diabetes  com¬ 
plications  sufficiently  to  be  reflected  in  ER  and  inpatient  care 
rates; 

•  some  of  the  demonstration  MTEs  already  had  been  working  on 
improving  diabetes  care  before  the  demonstration; 

•  the  TRICARE  Senior  Prime  MTEs  included  in  the  analysis  were 
not  fully  supported  by  RAND  and  MEDCOM; 

•  data  were  not  available  at  the  MEDCOM-level  for  the  measures 
targeted  by  the  MTEs’  action  plans; 

•  data  quality  issues  existed  for  patient  identifiers,  coding,  and 
clinical  laboratory  and  pharmacy  data; 

•  MEDCOM  lacked  centralized  support  for  data  acquisition  and 
monitoring. 


The  very  real  barrier  created  by  inadequate  availability  of  health¬ 
care  data  not  only  hinders  the  ability  to  measure  the  progress  of  the 
MTEs  in  diabetes  care  practice  improvements  but  also  weakens  the 
improvement  process  itself  by  depriving  the  MTEs  and  MEDCOM 
of  the  feedback  needed  to  guide  adjustments  to  the  quality  improve¬ 
ment  actions  being  taken  by  the  MTEs.  This  barrier  will  continue  to 
slow  progress  in  improving  practices  under  the  diabetes  guideline  as 
well  as  other  guidelines.  The  ability  of  MEDCOM  to  alleviate  the 
burden  on  its  MTEs  to  establish  a  valid  process  for  data  collection 
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and  monitoring  will  increase  the  likelihood  that  meaningful 
improvements  in  diabetes  care  will  he  achieved. 


Recommendations 

Although  the  MTFs  participating  in  the  diabetes  practice  guideline 
demonstration  had  some  notable  successes  in  some  aspects  of 
improving  diabetes  treatment  practices,  resource  limitations  and 
organizational  barriers  curbed  the  overall  progress.  Provided  here  are 
some  additional  lessons  learned  and  recommendations. 

Implementation 

•  Allow  for  flexibility:  Flexibility  in  implementation  strategies  can 
help  ensure  that  each  MTF  can  address  the  clinic  practices  most 
in  need  of  improvement  and  reflect  unique  capabilities,  but  it 
may  put  more  responsibility  on  each  MTF  for  defining  its  own 
direction,  and  it  also  may  slow  progress  toward  the  AMEDD 
goal  of  achieving  consistent  practices  across  its  facilities. 

•  Provide  and  ensure  adequate  resources:  Provision  of  additional 
resources,  including  regular  education  sessions  and  feedback  to 
providers,  to  support  implementation  activities  would  help  the 
champions  and  teams  achieve  lasting  improvements  in  practices. 

•  Learn  from  experience:  MEDCOM  should  continue  to  strength¬ 
en  its  system  in  response  to  the  lessons  identified  in  the  process 
evaluation  for  this  demonstration  as  well  as  its  experience  in 
previous  demonstrations. 

Benchmarking  of  MTF  Performance 

•  Measure  progress:  To  provide  an  empirical  foundation  to  guide 
performance  priorities,  MEDCOM  and  the  MTEs  should  use 
baseline  service  data  as  an  integral  part  of  the  regular  monitoring 
for  effective  diabetes  care  to  identify  facilities  at  greatest  variance 
from  established  standards  and  identify  factors  contributing  to 
the  variance.  Interventions  should  be  undertaken  to  correct 
identified  performance  problems. 


xxvi  Implementation  of  the  Diabetes  Practice  Guideline  in  AMEDD 


Outcomes  Measurement 

•  Document  variations:  MEDCOM  should  continue  to  document 
variations  in  performance  on  key  indicators  across  MTFs  on  a 
regular  basis  to  identify  areas  where  improvements  in  quality 
and  greater  consistency  are  needed. 

•  Select  indicators  and  apply  them  carefully:  It  is  important  to  insti¬ 
tute  a  set  of  indicators  that  are  widely  in  use  across  the  country, 
including  instructions  on  how  to  calculate  the  measures.  In 
addition,  careful  measurement  of  the  numerators  and  denomi¬ 
nators  for  performance  indicators  will  be  required  to  ensure 
effective  monitoring  of  progress. 

•  Educate  and  engage  providers  and  staff:  Educating  and  actively 
engaging  both  providers  and  clinic  staff  on  the  diabetes  practice 
guideline  can  help  achieve  sustainability  of  improved  practices. 

•  Include  patient  education  as  part  of  implementation:  Patient  edu¬ 
cation  is  an  important  aspect  of  diabetes  care,  especially  for  the 
new  diabetes  patient.  Further  assistance  by  MEDCOM  might 
be  useful  to  enhance  the  ability  to  reach  all  patients  and  offer 
comprehensive  education  for  managing  the  various  aspects  of 
their  diabetes. 

•  Provide  ongoing  monitoring  and  technical  support:  The  achieve¬ 
ment  of  sustainable  practice  improvements  can  be  encouraged 
by  MEDCOM  through  ongoing  monitoring  and  technical  sup¬ 
port  for  the  implementation  activities  of  the  Army  MTFs.  Also, 
to  successfully  introduce  and  consolidate  new  habits  among  a 
large  number  of  providers  and  clinic  staff,  implementation 
activities  require  not  only  resources  but  also  time  to  mature. 

•  Develop  a  patient  registry:  For  patients  with  chronic  conditions, 
such  as  asthma  or  diabetes,  a  registry  would  provide  a  centralized 
repository  of  pertinent  data  that  could  be  shared  by  all  MTFs  as 
the  patients  move  around  the  military  system.  Although 
AMEDD  does  not  have  centralized  registries,  many  of  the  local 
MTFs  are  attempting  to  establish  them  for  their  patient  popula¬ 
tions. 

•  Improve  centralized  data  collection:  Two  approaches  for  improve¬ 
ment  may  be  considered.  MEDCOM  could  establish  a  central- 
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ized  system  that  collects  the  data  directly  from  automated  data 
systems,  performs  analyses  in  the  central  office,  and  generates 
trend  reports  to  the  MTFs.  Alternatively,  the  system  could  use 
data  collected  and  analyzed  locally  hy  the  MTFs  and  reported  to 
MEDCOM,  which  then  would  aggregate  the  individual  MTF 
results  into  trend  reports. 


Costs 

•  Track  and  monitor  service  use  and  costs  o/’ faAjc.' MEDCOM 
should  continue  to  track  inpatient  use  rates  and  costs  over  time. 
As  cost  information  accumulates,  it  should  he  possible  to  distin¬ 
guish  trends  related  to  practice  changes  from  normal  fluctua¬ 
tions  in  health-care  needs  from  year  to  year. 
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CHAPTER  ONE 

Introduction 


The  Army  Medical  Department  (AMEDD)  has  made  a  commitment 
to  establishing  a  structure  and  process  to  support  its  military  treat¬ 
ment  facilities  (MTFs)  in  implementing  evidence-hased  practice 
guidelines  to  reduce  variation  and  enhance  quality  of  medical  care. 
Each  practice  guideline  is  a  statement  of  best  practices  for  the  man¬ 
agement  and  treatment  of  the  health  condition  it  addresses,  taking 
into  account  the  strength  of  relevant  scientific  evidence,  which  is 
documented  in  the  practice  guideline  report.  The  guidelines  support 
clinical  discretion  on  the  part  of  the  provider  while  identifying  spe¬ 
cific  practices  that  are  either  strongly  advised  or  not  advised. 

The  process  of  guideline  implementation  includes  monitoring 
the  effects  of  practice  improvements  on  clinical  care  outcomes.  With 
the  goal  of  establishing  implementation  and  monitoring  of  practice 
guidelines,  AMEDD  contracted  with  RAND  to  work  as  a  partner  in 
developing  and  testing  implementation  methods  for  ultimate  appli¬ 
cation  to  an  Army-wide  program  of  guideline-driven  practice. 

The  AMEDD/RAND  project  fielded  sequential  demonstrations 
over  a  two-year  period  (Figure  1.1)  to  test  implementation  of  clinical 
practice  guidelines  for  three  conditions:  lower  back  pain,  asthma,  and 
diabetes  mellitus.  This  approach  enabled  AMEDD  to  test  and  refine 
new  methods  on  a  small  scale  and  then  to  apply  these  methods  for 
rolling  out  use  of  practice  guidelines  across  the  Army  health  system. 

All  of  the  demonstrations  worked  with  practice  guidelines  that 
were  established  collaboratively  by  the  Departments  of  Veterans 
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Figure  1.1 

Diagram  of  the  Demonstration  Project 


Oct  '98  Jan  '99  Apr  '99  Jul  '99  Oct  '99  Jan  '00  Apr  '00  Jul  '00 

NOTE:  Two  MTFs  participated  in  the  diabetes  demonstration,  and  data  for  an 
additional  three  MTFs  that  also  implemented  this  guideline  were  used  in  the 
analysis  of  effects  of  implementing  the  guideline. 
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Affairs  (VA)  and  Defense  (DoD).  The  diabetes  guideline  demonstra¬ 
tion  was  the  last  of  the  three  demonstrations  and  was  implemented  by 
two  MTFs  in  AMEDD’s  Western  Region.  In  the  first  demonstration, 
four  MTFs  in  the  Great  Plains  Region  implemented  the  low  back 
pain  practice  guideline.  The  second  demonstration  evaluated  the 
implementation  of  the  asthma  guideline  by  four  MTFs  in  the  South¬ 
east  Region. 

RAND  performed  evaluations  for  each  demonstration:  the 
evaluations  included  an  assessment  of  the  implementation  process 
and  an  analysis  of  effects  on  clinical  practices  and  service  use.  The 
primary  purpose  of  the  evaluations  was  to  learn  from  the  experiences 
of  the  participating  MTFs,  with  respect  to  both  their  implementation 
processes  and  the  feasibility  and  data  requirements  for  measuring 
effects  of  the  practice  changes  they  introduced.  Thus,  many  aspects  of 
the  evaluation  were  exploratory,  and  this  report  documents  lessons 
learned  in  both  areas.  Specific  components  of  this  work  included: 

•  Implementation  process  evaluation — documented  the  imple¬ 
mentation  activities  of  participating  MTFs,  described  their  sue- 
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cesses  in  changing  clinical  practices,  identified  successes  and 
challenges  reported  hy  the  sites,  and  obtained  their  feedback 
regarding  MEDCOM  support. 

•  Analysis  of  effects  and  costs — estimated  the  extent  to  which 
the  sites’  implementation  activities  affected  specific  measures  of 
service  delivery  for  diabetes,  with  comparisons  to  a  control 
group  of  MTFs  that  did  not  implement  the  guideline,  as  well  as 
analysis  of  changes  in  costs  related  to  use  of  the  guideline. 

•  Benchmarking — described  variations  in  practices  across  MTFs 
for  the  measures  used  in  the  analysis  of  effects  to  help  identify 
priorities  for  future  interventions  and  for  comparing  individual 
facilities  to  benchmarks  for  target  levels  of  performance. 

•  Methods  development — documented  the  measurement  meth¬ 
ods  developed  and  the  related  data  requirements  to  provide  a 
basis  for  future  systemwide  monitoring  of  progress  in  achieving 
best  practices  for  each  condition  addressed  by  a  guideline. 

The  remainder  of  this  chapter  summarizes  the  process  DoD  and 
the  VA  used  to  establish  practice  guidelines  and  the  approach  used  by 
the  Army  Medical  Command  (MEDCOM)  for  implementing  the 
guidelines  in  the  Army  health  system. 


The  DoD/VA  Guideline  Adaptation  Process 

DoD  and  the  VA  initiated  a  collaborative  project  in  early  1998  to 
establish  a  single  standard  of  care  in  the  military  and  VA  health  sys¬ 
tems.  It  is  led  by  a  working  group  consisting  of  two  representatives 
from  the  Army,  Air  Force,  Navy,  and  VA.  The  goals  of  this  project 
are  adaptation  of  existing  clinical  practice  guidelines  for  selected 
conditions,  selection  of  two  to  four  indicators  for  each  guideline  to 
benchmark  and  monitor  implementation  progress,  and  integration  of 
DoD/VA  prevention,  pharmaceutical,  and  informatics  efforts. 

The  DoD/VA  Working  Group  designates  an  expert  panel  for 
each  practice  guideline  consisting  of  representatives  from  the  three 
military  services  and  the  VA  with  a  mix  of  clinical  backgrounds  tele- 
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vant  to  the  health  condition  of  interest.  The  expert  panel  reviews 
existing  national  guidelines  for  that  condition,  examines  and  updates 
the  scientific  evidence  supporting  the  guidelines,  and  adapts  the 
guidelines  for  the  military  and  VA  health  systems.  Each  panel  also 
develops  recommendations  for  the  metrics  to  he  used  to  monitor 
progress  in  guideline  implementation. 


Overview  of  the  Diabetes  Practice  Guideline 

The  principal  emphasis  of  the  DoD/VA  diabetes  practice  guideline  is 
on  effective  management  of  hlood  sugar,  with  the  goal  of  preventing 
short-term  and  long-term  complications  of  the  disease.  The  five  key 
elements  of  the  guideline  are  presented  in  Table  1.1. 

The  first  three  key  elements  are  the  core  procedures  for  diagno¬ 
sis  and  management  of  diabetes,  including  ongoing  patient  evalua¬ 
tion,  achievement  and  maintenance  of  glycemic  control,  and  patient 
education.  The  fourth  and  fifth  key  elements  address  early  detection 
and  management  of  diabetes-related  clinical  problems.  Procedures 
include  screening  for  elevated  blood  pressure,  eye  complications,  foot 
lesions,  elevated  cholesterol  or  lipids,  and  renal  disease,  all  of  which 
can  lead  to  life-threatening  complications  from  diabetes. 


Expected  Effects  on  Health-Care  Practices 

Any  change  in  clinical  practices  that  may  be  observed  as  MTFs  imple¬ 
ment  the  diabetes  guideline  should  reflect  the  guideline’s  emphasis  on 
effective  glycemic  control  and  patient  self-management  practices, 
coupled  with  regular  monitoring  for  diabetes-related  problems. 

A  set  of  performance  indicators  for  diabetes  care  has  been  devel¬ 
oped  through  the  Diabetes  Quality  Improvement  Project  (DQIP).' 


'  The  DQIP  is  sponsored  by  a  public/private  coalition  that  includes  the  American  Diabetes 
Association,  Foundation  for  Accountability,  Centers  for  Medicare  and  Medicaid  Services, 
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Table  1.1 

Key  Elements  of  the  DoD/VA  Diabetes  Practice  Guideline 


Key  Element 

Description 

Evaluation  for  Diabetes  Mellitus 

Evaluate  patient  for  existence 
and  type  of  diabetes  and 
stabilize  patient  for  diabetes 
management. 

Classify  patient  as  type  1  or  2  diabetic  and  identify 
and  document  comorbid  conditions.  Assess 
medical,  psychological,  and  social  stability.  Pro¬ 
vide  appropriate  treatment  and  stabilization 
based  on  these  assessments. 

Glycemic  Control 

Achieve  appropriate  glycemic 
control  by  assessing  and 
managing  glycosylated 
hemoglobin,  reported  as 
hemoglobin  A,c  (HbA,c)  lev¬ 
els. 

Assess  HbAjC  levels  relative  to  target  range.  If  level 
is  high,  check  for  patient  adherence  problems 
and  assess  need  to  adjust  glycemic  control  tar¬ 
get.  Provide  appropriate  interventions  to 
improve  patient  compliance,  adjust  medication 
therapy,  or  manage  side  effects,  including  con¬ 
traindications  to  treatment. 

Patient  Education 

Provide  education  for  new  and 
existing  diabetes  patients  to 
increase  disease  knowledge 
and  facilitate  self-care. 

Determine  patient's  extent  of  diabetes  knowledge 
and  self-management  skills  and  provide  educa¬ 
tion  as  needed  on  basic  concepts  and  core  com¬ 
petencies.  If  patient  needs  or  wants  further 
education,  provide  materials  or  refer  to  appro¬ 
priate  specialist  for  education  or  risk-focused 
intervention. 

Prevention  of  Complications 

Review  organ  systems  and  set 
priorities  for  patient's  care  to 
manage  problems  early  when 
they  occur  and  prevent  com¬ 
plications. 

Review  systems  regularly  to  detect  and  manage 
related  problems,  including  elevated  blood  pres¬ 
sure,  eye  exam  at  least  annually,  foot  risk 
assessment  or  lesions,  elevated  cholesterol  or  lip¬ 
ids,  and  renal  disease  (albuminuria  or  elevated 
creatinine). 

Management  of  End-System  Involvement 

Manage  treatment  for  end- 
system  involvement  when 
necessary  through  regular 
care  and  specialty  referrals,  as 
appropriate. 

When  related  problems  are  identified,  treat  them 
as  indicated  and  consider  specialty  referral  to 
manage  serious  cases  or  secondary  causes.  Coun¬ 
sel  patient  on  self-care  and  lifestyle  modifica¬ 
tions,  reinforcing  advice  in  follow-up.  Continue 
to  manage  status  of  the  problems  at  each  office 
visit. 

National  Committee  for  Quality  Assurance,  American  Academy  of  Family  Physicians, 
American  College  of  Physicians,  DoD,  and  the  VA. 
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These  measures  were  adopted  by  the  DoD/VA  Working  Group  as 
the  official  metrics  to  be  monitored  for  its  diabetes  practice  guideline. 
Adoption  of  a  practice  guideline  based  on  these  measures  predicts  a 
number  of  changes  in  clinical  practice  (Table  1.2). 

Table  1.2 

Changes  in  Clinical  Practices  Predicted  by  Practice  Guideline  Implementation 


Initial  Assessment  and  Glycemic  Control 

Increased  rates  of  primary  care  clinic  visits  for  diabetes  patients  during  the  first 
quarter  of  practice  guideline  implementation,  followed  by  a  decline  in  visit  rates 
during  subsequent  quarters 

Smaller  increases  in  primary  care  clinic  visits  for  patients  not  being  treated  with 
insulin  therapy,  compared  with  patients  using  insulin  because  of  visit  frequency 
involved  in  adjusting  insulin  dosages 

Increase  in  the  percentage  of  noninsulin  patients  who  fill  prescribed  medications 
to  control  HbA,c  levels 

Increased  referrals  for  diabetes  education  services 

Increased  percentages  of  patients  with  at  least  one  test  for  glycosylated  hemo¬ 
globin,  reported  as  hemoglobin  A,c  (HbA,cy 
Increased  number  of  HbA,c  level  tests  per  diabetes  patient 
Larger  increases  in  frequency  of  HbA,c  level  testing  for  patients  who  are  not 
being  treated  with  insulin  therapy,  compared  with  patients  using  insulin 
Decrease  in  average  levels  of  HbA,c  for  diabetes  patients® 

Decreased  percentage  of  patients  with  HbA,c  at  greater  than  9.5  percent® 
Decreased  variation  across  patients  in  average  levels  of  HbA,c® 

Evaluation  and  Prevention  of  Diabetic  Complications 

Increased  percentage  of  patients  assessed  for  nephropathy® 

Increased  percentage  of  patients  receiving  a  lipid  profile  in  a  year'’ 

Increased  percentage  of  patients  with  a  low-density  lipoprotein  (LDL)  (less  than 
130  mg/dL®) 

Increased  percentage  of  patients  receiving  a  dilated  eye  exam  at  least  annually® 
Increased  percentage  of  primary  care  visits  at  which  patients  receive  a  foot  exam® 

Management  of  Avoidable  Hypo-  or  Hyperglycemic  Episodes 

Decreased  number  of  emergency  room  (ER)  visits  for  diabetes  patients  due  to 
hypo-  or  hyperglycemia 

Decreased  rates  of  inpatient  admission  for  hypo-  or  hyperglycemia  following  ER 
visits 

Decreased  number  of  total  hospitalizations  for  diabetes  patients 
Increased  number  of  diabetes  patients  with  a  primary  care  visit  after  a  hospital 
discharge 

Increased  frequency  of  HbA,c-level  tests  in  the  quarter  following  an  ER  visit  or 
hospital  stay 

®These  changes  are  also  included  in  DQIP's  recommendations. 

'’DQIP  Guidelines  call  for  an  increase  in  two  years. 
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A  Systems  Approach  to  Implementation 

Most  studies  that  have  evaluated  the  effects  of  guideline  implementa¬ 
tion  on  health-care  practices  have  been  fairly  narrow  assessments  of 
individual  interventions  to  change  provider  behavior  (e.g.,  education, 
audit  and  feedback,  reminders)  primarily  because  of  researchers’ 
efforts  to  design  studies  with  effective  controls.  Results  across  studies 
are  quite  variable,  explained  partly  by  differences  in  the  subject  mat¬ 
ter  of  the  guideline,  provider  attitudes,  and  organizational  characteris¬ 
tics  (Grilli  and  Lomas,  1994;  Chodoff  and  Crowley,  1995;  Paradox, 
1995;  Eastwood  and  Sheldon,  1996).  The  results  are  often  disap¬ 
pointing,  as  in  the  finding  that  nearly  one-third  of  the  time  primary 
care  providers  fail  to  follow  even  noncontroversial  and  evidence-based 
guideline  recommendations  (Grol  et  ah,  1998).  Active  methods,  such 
as  concurrent  reminders  and  academic  detailing,  are  more  consis¬ 
tently  effective  than  passive  dissemination  of  guidelines  or  feedback. 
Combining  two  or  more  approaches  seems  more  likely  to  succeed 
than  relying  on  a  single  intervention,  with  multifaceted  interventions 
targeted  at  identified  barriers  being  the  most  successful  (Bero  et  ah, 
1998;  Grimshaw  et  al.,  2001). 

Influenced  by  systems  thinking  and  quality  improvement,  health 
care  managers  favor  multifaceted  changes  in  systems,  rather  than  sin¬ 
gle  interventions,  as  the  best  hope  for  changing  patient  care  practices 
(Senge,  1990;  Shortell,  Bennett,  and  Byck,  1998).  The  Chronic  Care 
Model,  for  example,  is  testing  the  assumption  that  care  of  the  chroni¬ 
cally  ill  requires  major  changes  in  the  organization  and  delivery  of 
care,  in  information  systems,  in  doctor-patient  relationships,  in 
patient  self-management,  and  even  in  relationships  between  the 
health  system  and  community  resources  (Wagner,  Austin,  and 
VonKorff,  1996;  Von  Korff  et  al.,  1997).  A  premise  of  this  and  other 
integrated  models  is  that  testing  the  effects  of  individual  components 
will  yield  misleading  null  results  because  dramatic  changes  in  out¬ 
come  only  occur  when  all  components  of  the  model  are  in  place. 

An  additional  consideration  is  the  distinction  between  provider- 
controlled  and  system-dependent  guideline  criteria  (Hargraves  et  al., 
1996).  Because  systems  changes  (such  as  computerized  order  entry 


8  Implementation  of  the  Diabetes  Practice  Guideline  in  AMEDD 


linked  to  decision  support)  can  clearly  change  the  degree  of  compli¬ 
ance  with  practitioner-controlled  criteria  (such  as  choice  of  antibi¬ 
otic),  determining  the  dominant  influence  on  practice  is  difficult 
(Evans  et  ah,  1998). 

Basic  Implementation  Strategy 

The  AMEDD  practice  guideline  implementation  demonstrations 
applied  a  systems  approach.  This  approach  was  amply  supported  hy 
lessons  from  the  earlier  demonstrations,  which  documented  the 
importance  of  addressing  multiple  factors  influencing  clinical  prac¬ 
tices.  These  demonstrations  highlighted  that  two  main  dimensions 
need  to  he  addressed  to  ensure  successful  changes  in  practices  hy 
MTEs  and  other  local  facilities:  huild  local  ownership  or  “huy-in” 
from  the  staff  responsible  for  implementing  the  new  practices  and 
ensure  that  clinical  and  administrative  systems  are  in  place  to  facili¬ 
tate  staff  adherence  to  the  guideline. 

Eigure  1.2  presents  a  model  of  how  staff  buy-in  and  system 
changes  can  interact  to  produce  different  implementation  results. 
Having  both  local  ownership  and  system  support  produces  the  opti¬ 
mal  result,  leading  to  likely  success.  System  support  without  local 
ownership  produces  providers  resistant  to  implementation,  despite 
having  clinic  procedures  and  systems  equipped  to  support  the  pro¬ 
cess.  Provider  ownership  without  system  support  produces  providers 
who  wish  to  change  practices  but  are  frustrated  by  their  inability  to 


Figure  1.2 

Matrix  of  Implementation  Outcomes 

Local  ownership  No  local  ownership 


Systems  do  support 
recommended  practices 


Systems  do  not  support 
recommended  practices 


/ 

Provider  resistance 

Frustrated  providers 

X 
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overcome  barriers  in  the  MTF  systems  that  hamper  their  ability  to  do 
so.  Finally,  with  neither  local  ownership  nor  system  support,  imple¬ 
mentation  will  fail. 

Six  Critical  Success  Factors 

Drawing  on  published  literature  and  the  experiences  observed  in  the 
AMEDD  demonstrations,  we  identified  six  critical  success  factors 
that  strongly  influence  how  successfully  an  MTF  will  be  able  to  inte¬ 
grate  new  practices  into  its  clinical  and  administrative  processes: 

•  Visible  and  consistent  commitment  by  command  leadership  at 
the  MTF,  regional,  and  corporate  levels,  including  both  state¬ 
ments  establishing  a  priority  for  the  work  and  actions  that 
support  those  statements.  Such  support  has  been  shown  to  be 
necessary  to  empower  teams  to  change  practices  effectively  (Sol- 
berg  et  ah,  1997;  Keller,  1997;  Motwani,  Klein,  and  Navitskas, 
1999;  Savitz  and  Kaluzny,  2000). 

•  Ongoing  monitoring  of  progress  in  carrying  out  an  implementa¬ 
tion  plan,  to  be  performed  by  both  the  MTFs  and  the  Army 
Medical  Command  (MEDCOM),  with  regular  feedback  to  the 
MTEs  on  the  effects  of  their  actions  on  desired  outcomes 
(Palmer  and  Hargraves,  1996;  Sasala  and  Jasovsky,  1998;  Cox  et 
al.,  1999;  Eescoe-Eong  and  Eong,  1999;  Savitz  and  Kaluzny, 
2000). 

•  Provision  of  implementation  guidance  and  support  to  the  MTEs 
by  MEDCOM,  including  toolkits  of  support  materials  and 
ready  access  to  staff  support  and  other  resources.  Such  support 
encourages  MTEs  to  make  needed  practice  changes  to  move 
toward  consistency  in  practices  across  the  Army  facilities  (Sasala 
and  Jasovsky,  1998;  Motwani  et  al.,  1999). 

•  Identification  of  a  physician  at  each  MTE  who  is  a  respected 
local  opinion  leader  to  serve  as  guideline  champion  and  lead  the 
MTE’s  implementation  activities  (Palmer  and  Hargraves,  1996; 
Solberg  et  al.,  1997;  Gandhi  et  al.,  2000). 

•  Provision  of  adequate  dedicated  time  and  other  resource  support 
for  the  guideline  champions  to  enable  them  to  perform  their 
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tasks  effectively.  Such  support  also  will  reinforce  the  signals  that 
guideline  implementation  is  a  priority  for  the  MTF  command 
(Palmer  and  Hargraves,  1996;  Lescoe-Long  and  Long,  1999; 
Gandhi  et  al.,  2000). 

•  Institutionalization  of  new  practices  as  part  of  a  clinic’s  normal 
(routine)  procedures  within  a  finite  period.  This  requires  suc¬ 
cessful  design  and  execution  of  an  action  plan  to  change  prac¬ 
tices,  including  both  educational  and  systems  change 
interventions  (Solherg  et  ah,  1997;  Motwani  et  ah,  1999). 

The  AMEDD  Guideline  Implementation  Process 

The  systems  approach  applied  in  the  demonstrations  to  implement 
the  practice  guidelines  is  shown  in  Figure  1.3.  This  process  consisted 
of  the  following  components: 

•  Practice  guideline  and  metrics.  The  official  DoD/VA  practice 
guideline  materials  are  provided  to  the  MTFs,  including  a  sum¬ 
mary  list  of  the  key  elements  of  the  guideline  and  metrics  identi¬ 
fied  by  the  guideline  expert  panel  for  monitoring  progress. 

•  Guideline  toolkit.  The  MEDCOM  and  the  Center  for  Health 
Promotion  and  Preventive  Medicine  (CHPPM)  collaborated  in 
the  development  of  a  toolkit  of  materials  to  support  the  MTFs’ 
guideline  implementation  activities  (e.g.,  documentation  forms, 
provider  training  videos,  patient  education  materials,  reminder 
cards).  Toolkits  are  provided  to  each  of  the  demonstration 
MTFs,  and  consumable  items  are  replenished  as  needed.  Toolkit 
contents  were  revised  based  on  demonstration  MTF  feedback. 

•  Kickoff  planning  conference.  Multidisciplinary  teams  from  the 
demonstration  MTFs  participate  in  a  two-day  interactive  meet¬ 
ing  to  develop  their  guideline  implementation  strategies  and 
action  plans.  Interaction  is  within  and  between  teams  and  with 
RAND  and  MEDCOM  facilitators. 

•  MTF  implementation  activities.  Following  the  kickoff  confer¬ 
ence,  the  MTF  teams  carry  out  their  action  plans.  They  prepare 
monthly  reports  that  summarize  their  recent  activities,  successes, 
challenges,  and  assistance  needed  to  support  their  work. 
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Figure  1.3 

Guideline  Implementation  Process 
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•  Information  exchange.  Teams  are  encouraged  to  share  their 
experiences  with  each  other  and  with  RAND  and  MEDCOM 
facilitators  so  they  can  learn  from  errors  and  huild  on  successes. 
Communication  occurs  through  MEDCOM-supported  listservs 
(mailing  list  programs  for  communicating  with  other  people 
who  have  suhscrihed  to  the  same  list)  and  direct  e-mail. 

•  Monitoring  of  progress.  Monitoring  of  implementation  pro¬ 
gress  is  performed  hy  both  MEDCOM  and  the  participating 
MTFs  through  site  visits  and  data  monitoring.  Monitoring  at 
the  MEDCOM  level  focuses  on  metrics  that  have  been  devel¬ 
oped  in  the  DoD/VA  guideline  process.  MTF  monitoring  uses 
either  DoD/VA  guideline  metrics  or  guideline  implementation 
process  metrics.  The  MTFs  were  encouraged  to  establish  mea¬ 
sures  for  their  key  strategies  to  enable  them  to  assess  their  pro¬ 
gress  in  making  the  clinical  process  changes  they  intended. 


The  Demonstration  Sites 

The  diabetes  demonstration  was  carried  out  in  the  Army’s  Western 
Region.  The  Western  Region  was  selected  for  the  third  and  final 
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demonstration  because  MTFs  in  that  region  had  not  yet  had  training 
in  the  AMEDD  guideline  implementation  system  and  methods,  and 
one  MTF  (Madigan  Army  Medical  Center  [AMC])  was  known  to 
have  a  large  number  of  diabetic  patients. 

Two  MTFs  in  the  Western  Region — Madigan  AMC,  Fort  Few- 
is,  Washington,  and  Bassett  Army  Community  Hospital  (ACH),  Fort 
Wainwright,  Alaska — served  as  the  demonstration  sites.  These  two 
MTFs  offer  a  strong  contrast  between  a  large,  specialty  medical  center 
in  an  urbanized  area  (Madigan)  and  a  small  community  hospital  at  a 
remote  military  post  (Bassett).  Madigan  AMC,  the  largest  military 
hospital  on  the  west  coast,  has  222  inpatient  beds  and  a  full  range  of 
primary  and  specialty  care  capabilities.  It  serves  as  the  regional  referral 
center  for  smaller  MTFs  and  clinics  operated  by  the  Army,  Air  Force, 
and  Navy.  Bassett  ACH  is  a  43-bed  hospital  at  Fort  Wainwright  near 
Fairbanks,  the  northernmost  Army  installation.  These  two  MTFs  also 
serve  very  different  sizes  and  mixes  of  beneficiary  populations.  As 
shown  in  Table  1.3,  Madigan’s  catchment  area  contains  an  estimated 
92,000  beneficiaries,  of  whom  31.6  percent  are  retirees.  About  2  per¬ 
cent  of  Madigan’s  beneficiaries  are  diabetic  patients.  Bassett  ACH 
serves  about  22,000  beneficiaries  north  of  the  Alaska  Range  through 
the  hospital  and  several  outlying  clinics.  Of  Bassett’s  22,000  benefici¬ 
aries,  only  15.7  percent  are  retirees.  Fess  than  2  percent  of  benefici¬ 
aries  at  Bassett  ACH  are  diabetes  patients. 

Simultaneous  with  the  start  of  the  AMEDD  demonstration, 
three  other  Army  MTFs  implemented  the  diabetes  guideline  as  part 


Table  1.3 

Profiles  of  the  Military  Treatment  Facilities  Participating  in  the  Diabetes 
Guideline  Demonstration 


Type  of  Beneficiaries 

Madigan  AMC,  Fort 
Lewis 

Bassett  ACH,  Fort 
Wainwright 

Number 

Percentage 

Number 

Percentage 

Active-duty 

22,878 

24.9 

7,662 

39.9 

Active-duty  family  members 

39,966 

43.5 

8,515 

44.4 

Retirees,  family,  survivors 

28,978 

31.6 

3,008 

15.7 

Total 

91,822 

100.0 

19,185 

100.0 
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of  the  Medicare-DoD  Senior  Prime  demonstration.  Because  some 
data  from  the  guideline  implementation  at  these  MTFs  were  included 
in  RAND’s  analysis  of  guideline  effects,  the  facilities  are  briefly 
described  here.  The  MTFs  were  Brooke  AMC  at  Fort  Sam  Houston 
in  Texas,  Evans  ACH  at  Fort  Carson  in  Colorado,  and  Reynolds 
ACH  at  Fort  Sill  in  Oklahoma.  Brooke  AMC  is  a  450-bed  medical 
facility  that  includes  operating  rooms,  oral  surgery  suites,  dental 
treatment  rooms,  a  diagnostic  and  therapeutic  radiology  center,  a 
same-day  surgery  suite,  and  the  requisite  outpatient  clinics  and  ancil¬ 
lary  support  services.  Evans  ACH  is  a  195-bed  facility  with  medical 
and  surgical  inpatient  units  and  primary  and  specialty  care  outpatient 
services.  Reynolds  ACH  is  a  150-bed  hospital  that  provides  primary 
care  services  through  several  clinics,  as  well  as  a  range  of  specialty  care 
and  ancillary  treatment  services. 


The  RAND  Evaluation 

RAND’s  evaluation  of  the  demonstration  consisted  of  two  compo¬ 
nents.  The  first  component  was  an  evaluation  of  the  implementation 
process  at  two  AMEDD  sites  and  two  of  the  three  TRICARE  Senior 
Prime  sites  using  site  visits  and  interviews.  The  site  visits  to  the 
TRICARE  Senior  Prime  sites  were  conducted  by  the  Region  6 
(which  covers  Texas  and  much  of  the  Southwest)  head  Agent’s  office, 
using  RAND’s  evaluation  model.  The  second  component  was  an 
analysis  of  the  effects  of  the  guideline  on  service  utilization  at  the  two 
AMEDD  sites  and  three  TRICARE  Senior  Prime  sites.  Baseline 
service  utilization  (prior  to  guideline  implementation)  was  also 
assessed  for  comparison  and  to  establish  a  benchmark  for  current 
practice.  Service  utilization  was  also  assessed  at  five  control  sites  to 
rule  out  the  effects  of  temporal  trends.  Included  in  the  analysis  of 
service  utilization  was  an  assessment  of  the  adequacy  of  Army  medical 
databases  for  monitoring  the  results  of  the  guideline  implementation 
as  well  as  future  follow-up  and  provider  feedback.  The  impact  of  the 
guideline  on  costs  was  also  assessed. 
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Organization  of  This  Report 

In  the  remainder  of  the  report,  we  present  our  evaluation  methods 
and  findings.  Chapter  Two  describes  the  methods  and  data  used  for 
the  evaluation.  Chapter  Three  provides  information  on  the  size  and 
characteristics  of  the  diabetes  population  served  by  Army  MTFs  and 
profiles  baseline  performance  for  the  MTFs  included  in  the  evalua¬ 
tion  on  each  of  the  measures  used  to  assess  the  effects  of  the  guideline 
on  clinical  practices  for  diabetes  patients.  Results  of  the  process 
evaluation  are  reported  in  Chapter  Four,  and  results  of  the  evaluation 
of  guideline  effects  are  presented  in  Chapter  Five.  In  Chapter  Six,  we 
synthesize  the  results  of  the  full  evaluation  and  identify  lessons 
learned  and  implications  for  systemwide  guideline  implementation 
strategies  and  also  include  our  recommendations. 


CHAPTER  TWO 

Methods  and  Data 


The  RAND  evaluation  of  the  diabetes  guideline  demonstration 
included  two  components:  an  assessment  of  the  implementation 
process  and  an  assessment  of  the  effects  of  these  implementation 
activities  on  delivery  of  care  for  diabetic  patients.  In  this  chapter,  we 
summarize  the  methods  and  data  for  these  two  evaluation  compo¬ 
nents.  Additional  details  are  provided  in  Appendix  A. 

Implementation  of  a  clinical  practice  guideline  is  one  type  of 
quality  improvement  intervention.  An  evaluation  of  any  quality 
improvement  intervention  should  recognize  the  incremental  nature  of 
the  processes  of  change,  which  require  time  to  achieve  lasting  and 
measurable  practice  improvements.  Therefore,  a  comprehensive  eval¬ 
uation  of  guideline  implementation  would  recognize  and  assess  three 
phases: 

•  Introducing  new  practices.  Evaluating  initial  practice  imple¬ 
mentation  efforts  emphasizes  documentation  of  the  extent  to 
which  effective  action  plans  are  developed  and  the  intended 
actions  are  actually  implemented  using  process  evaluation  meth¬ 
ods  and  (sometimes)  feedback  to  participants. 

•  Achieving  intended  changes  in  practices.  Evaluating  process 
change  involves  monitoring  short-term  effects  on  service- 
delivery  methods  and  activities  using  relevant,  quantifiable  out¬ 
come  measures. 

•  Improving  patient  (clinical)  outcomes.  Assessing  patient  out¬ 
comes  is  a  longer-term  effort  and  also  uses  relevant,  quantifiable 
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outcome  measures.  Patient  outcomes  were  not  assessed  in  this 
effort. 

The  data  collected  hy  RAND  for  the  second  phase  of  assessment 
were  also  used  to  assess  the  availahility  and  usefulness  of  DoD 
administrative  data  for  monitoring  effects  of  practice  improvement 
processes  on  clinical  practices.  In  this  analysis,  we  assessed  which 
indicators  could  he  measured  using  centrally  available  (administra¬ 
tive)  data  and  which  indicators  required  data  currently  available  only 
at  the  MTFs.  We  also  examined  coding  and  measurement  issues  that 
must  be  addressed  to  establish  valid  measures  of  the  indicators  using 
the  administrative  data.  We  document  our  findings  on  these  issues  in 
Chapter  Four. 


Implementation  Process  Evaluation  Methods 

To  learn  from  the  experience  of  the  MTFs  participating  in  the  dem¬ 
onstration,  the  RAND  team  used  a  participant-observer  approach  to 
exchange  information  and  facilitate  shared  learning  with  the  MTFs 
throughout  the  demonstration  and  evaluation  process.  The  purposes 
of  the  implementation  process  evaluation  were  to 

•  document  the  actions  and  experiences  of  the  demonstration 
MTFs  with  practice  guideline  implementation  and  assess  per¬ 
formance  relative  to  each  of  the  six  key  success  factors  described 
in  Chapter  One; 

•  identify  areas  where  the  policies,  systems,  and  processes  estab¬ 
lished  by  the  AMEDD  for  guideline  implementation  can  be 
strengthened;  and 

•  assess  the  degree  to  which  demonstration  sites  might  be  able  to 
build  on  their  experiences  with  the  demonstration  guideline  to 
implement  additional  DoD/VA  guidelines. 

Information  was  collected  from  the  participating  MTFs  through 
a  series  of  site  visits,  monthly  progress  reports  prepared  by  partici- 
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paring  MTFs,  and  questionnaires  completed  by  individual  partici¬ 
pants.  Additional  details  about  these  methods  can  be  found  in 
Appendix  A. 

Evaluation  Site  Visits 

Three  site  visits  were  conducted  at  each  demonstration  site:  an  intro¬ 
ductory  visit  before  the  kickoff  conference  and  two  postimplementa¬ 
tion  visits  scheduled  for  the  third  and  tenth  month  after  the  MTFs 
began  implementing  their  action  plans.  The  demonstration  and  eval¬ 
uation  schedule  was  as  follows: 


•  November  1999 — introductory  site  visits  to  demonstration 
MTFs. 

•  December  1 999 — kickoff  planning  meeting  for  MTFs. 

•  April  2000 — first  evaluation  site  visits  three  months  after  the 
MTFs  began  implementing  their  plans. 

•  October  2000 — second  evaluation  site  visits  the  tenth  month  of 
implementation. 

In  preparation  for  the  site  visits,  RAND  developed  an  agenda  of 
the  group  meetings,  individual  interviews,  and  focus  groups  that  we 
wanted  to  conduct.  The  facilitator  of  each  implementation  team 
worked  with  that  agenda  to  schedule  the  meetings  with  implementa¬ 
tion  team  members  and  other  individuals  involved  in  the  implemen¬ 
tation  process. 

MEDCOM  staff  participated  in  the  site  visits  with  the  RAND 
evaluation  team,  which  allowed  MEDCOM  to  learn  directly  from 
the  MTEs’  experiences.  RAND  conducted  the  interviews  and  focus 
groups  and  the  MEDCOM  staff  provided  technical  assistance  and 
other  support  to  the  MTE  teams.  At  the  conclusion  of  each  evalua¬ 
tion  visit,  we  briefed  the  MTE  command  group  about  what  we  had 
learned  and  issues  identified,  which  is  a  standard  step  for  site  visit 
processes  at  military  facilities. 

During  the  postimplementation  site  visits,  we  interviewed  the 
guideline  champion,  team  facilitator,  and  implementation  team 
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members  to  learn  their  respective  perspectives  on  the  process  and 
their  experiences.  Semistructured  interview  methods  were  used  for  all 
interviews  and  group  discussions,  working  from  predefined  lists  of 
questions  to  cover  during  each  session.  All  individuals  interviewed  or 
in  the  focus  groups  were  informed  that  the  interviews  were  voluntary 
and,  to  protect  their  privacy,  everything  we  reported  from  the  evalua¬ 
tion  would  be  designed  so  that  individuals  could  not  be  identified  as 
sources  of  comments  or  observations  reported. 

Focus  groups  were  conducted  with  three  stakeholder  groups:  the 
implementation  team  members,  providers,  and  other  clinic  staff.  All 
of  the  MTF  champions  and  facilitators  invited  all  individuals  in  each 
stakeholder  group  to  participate  in  the  focus  groups.  To  give  focus 
group  participants  the  privacy  to  express  their  opinions  freely,  the 
implementation  team  did  not  attend  the  focus  groups  for  providers  or 
other  clinic  staff  At  the  start  of  each  focus  group,  we  advised  the  par¬ 
ticipants  of  the  informed-consent  provisions.  We  also  described  how 
this  focus  group  fit  into  the  overall  site  visit  process,  including  the 
final  outbriefing  to  the  MTF  command. 

Using  a  written  protocol,  we  asked  participants  in  each  focus 
group  questions  regarding  their  attitudes  toward  guideline  implemen¬ 
tation,  how  they  worked  with  the  practice  guideline,  how  they  were 
affected  by  the  implementation  process,  and  issues  or  concerns  they 
had  identified.  There  was  good  attendance  and  active  discussion  in 
the  focus  groups,  giving  us  confidence  that  the  feedback  from  the 
groups  represented  the  sense  of  the  various  stakeholder  groups. 

Finally,  we  interviewed  the  command  leadership  of  the  MTF. 
These  sessions  allowed  us  to  communicate  issues  to  the  leadership 
and  to  obtain  their  feedback  on  the  practice  guideline,  the  implemen¬ 
tation  process,  and  the  level  of  priority  placed  on  the  work  by  the 
leadership. 

Other  Information  Collection  Activities 

At  the  second  postimplementation  site  visits  (ten  months  into 
implementation),  we  began  each  interview  or  focus  group  by  asking 
the  participants  to  complete  a  written  questionnaire  that  gathered 
information  on  individuals’  views  of  the  guideline  and  experiences  in 
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the  implementation  process.  Separate  questionnaires  were  used  for 
each  of  three  groups:  providers,  other  clinic  staff,  and  the  implemen¬ 
tation  team  members.  Because  the  number  of  respondents  was  small 
for  each  group,  we  could  not  analyze  or  interpret  the  results  statisti¬ 
cally.  However,  the  survey  information  offered  some  insights  into 
issues  and  experiences  that  were  useful  additions  to  the  process 
evaluation. 

The  final  source  of  process  evaluation  information  was  monthly 
progress  reports  prepared  by  the  participating  MTFs  and  submitted 
to  RAND.  These  reports  provided  us  valuable  information  on 
implementation  progress  over  time,  which  we  used  in  our  preparation 
for  the  second  postimplementation  site  visits.  They  also  stimulated 
action  by  both  the  MTFs  and  MEDCOM  because  the  MTFs  identi¬ 
fied  issues  requiring  resolution. 


Outcome  Evaluation 

The  outcome  analysis  had  four  goals: 

•  Document  the  changes  in  clinical  processes  and  service  activity 
in  the  programs  that  implemented  the  practice  guideline. 

•  Document  changes  in  diabetic  patients’  service  utilization  that 
are  attributable  to  the  clinical  process  changes  that  have 
occurred. 

•  Assess  average  MTF  costs  of  care  for  treatment  of  the  diabetic 
patients  they  serve  and  how  those  costs  may  have  been  affected 
by  implementation  of  the  guideline. 

•  Examine  the  usefulness  of  the  metrics  and  measurement  meth¬ 
ods  used  in  the  demonstration  and  implications  for  how  best  to 
establish  an  effective  system  for  routine  monitoring  of  ongoing 
progress. 

Chapter  One  provided  an  exhaustive  list  of  service  outcomes 
that  could  be  expected  from  implementation  of  a  diabetes  guideline. 
Aware  that  we  could  not  assess  all  these  potential  changes,  we  identi- 
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fled  a  set  of  metrics  that  would  provide  a  perspective  broad  enough  to 
allow  priority-setting  for  future  quality  and  performance  monitoring 
activities  and  stimulate  improvements  in  data  collection  capabilities 
to  permit  measurement  of  important  indicators  of  quality  care.  The 
hypotheses  that  generated  our  outcome  measures  are  listed  in  Table 
2.1.  The  outcome  measures  are  discussed  later  in  this  chapter. 

Evaluation  Design 

To  test  these  hypotheses,  we  used  an  interrupted  time  series  control- 
group  design.  Trends  for  the  outcome  measures  were  estimated  for 
one  year  before  the  demonstration  sites  began  using  the  diabetes 
guideline  and  for  one  year  after  its  start  date.  This  approach  allowed 
us  to  estimate  annual  utilization  measures,  and  it  also  controlled  for 
any  seasonal  effects  on  diabetes  care.  A  control  group  of  five  addi¬ 
tional  MTFs  was  used  to  adjust  for  underlying  historical  trends  that 
otherwise  could  threaten  the  validity  of  findings  at  the  demonstration 
MTFs.  The  evaluation  design  is  shown  in  Figure  2.1. 

Choice  of  Demonstration  and  Control  Sites 

The  demonstration  sites  were  described  in  Chapter  One.  The  five 
control  MTFs  were  selected  by  MEDCOM  and  the  Army’s  Patient 
Administration  Systems  and  Biostatistical  Activity  (PASBA)  to  pro¬ 
vide  a  reasonable  match  to  the  demonstration  sites,  based  on  peer 


Table  2.1 

Expected  Effects  of  Proactive  Diabetes  Care  Management 


Short-term  increases  in  the  number  of  primary  care  visits  per  diabetic  patient  fol¬ 
lowed  by  a  decline  to  stable  visit  rates  and,  early  in  the  demonstration,  a  larger 
increase  in  the  number  of  primary  care  visits  per  insulin-dependent  patient  than  for 
those  who  are  not  insulin-dependent 

Increase  in  the  percentage  of  non-insulin  dependent  diabetic  patients  on  oral  hypo¬ 
glycemic  agents 

Increase  in  the  percentage  of  diabetic  patients  who  had  at  least  one  eye  exam  annu¬ 
ally 

Decrease  in  the  number  of  ER  visits  per  diabetic  patient 

Decrease  in  the  number  of  hospitalizations  per  diabetic  patient 
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Figure  2.1 

Evaluation  Timeline 
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NOTE:  B  =  baseline  period  (study  year  one);  E  =  experimental  period  (study  year  two); 
C  =  Control  site  conditions  (no  guideline  introduction). 
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groupings  already  established  for  benchmarking  within  AMEDD 
(based  on  size  and  service  mix)  and  to  ensure  inclusion  of  sites  that 
did  not  undertake  initiatives  to  improve  care  for  diabetic  patients 
during  the  two-year  study  period. 

Data  Sources 

The  analyses  conducted  in  this  study  required  data  on  outpatient  vis¬ 
its  to  primary  care  providers,  visits  to  hospital  ERs,  hospital  inpatient 
stays,  use  of  medications  for  glycemic  control,  eye  exams,  and  labora¬ 
tory  tests.  Table  2.2  shows  the  sources  of  the  data. 

All  of  these  data  except  the  MEPRS  data  were  extracted  by 
PASBA,  and  the  extracted  data  hies  were  transmitted  to  RAND  for 
analysis.  Unit  cost  estimates  based  on  the  MEPRS  data  had  been 
obtained  by  RAND  directly  from  DoD  as  part  of  its  evaluation  of  the 
Medicare-DoD  Subvention  Evaluation.  Details  of  the  methods  for 
extracting  data  from  these  sources  and  for  construction  of  the  analysis 
files  are  presented  in  Appendix  A. 

The  Diabetic  Population 

The  total  diabetic  population  ultimately  addressed  in  this  study  con¬ 
sisted  of  all  individuals  who  used  an  Army  MTF  for  diabetes  care  at 
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Table  2.2 

Sources  of  Data  for  Analyses 


Data  Type 

Source 

MTF  outpatient  visits 

Standard  Ambulatory  Data  Record  (SADR)  database 
extracted  from  MTF  Ambulatory  Data  System  (ADS) 
data 

MTF  inpatient  stays 

Standard  Inpatient  Data  Record  (SIDR)  database 
extracted  from  MTF  Composite  Health-Care  System 
(CHCS)  data 

Outpatient  visits  and 
inpatient  stays  for  net¬ 
work  providers 

Health-Care  Service  Records  (HCSR)  database  main¬ 
tained  in  the  TRICARE  Civilian  Health  and  Medical 
Program  of  the  Uniformed  Services  (CHAMPUS)  data 
system 

MTF  pharmacy  data 

Uniformed  Services  Prescription  Database  (USPD) 
maintained  by  the  PharmacoEconomic  Center 

National  Mail  Order  Phar¬ 
macy  (NMOP)  data 

TRICARE  NMOP  data  system 

Laboratory  tests 

Triservice  query  of  CHCS  clinical  chemistry  laboratory 
data  performed  collaboratively  by  the  Air  Force's 
Population  Health  and  Safety  Division  (PHSD), 

PASBA,  and  the  National  Maritime  Intelligence  Cen¬ 
ter 

Financial  data 

Medical  Expense  and  Performance  Reporting  System 
(MEPRS)= 

“  The  DoD  financial  management  data  system  that  maintains  facility-level  financial 
data  for  all  MTFs.  This  system  uses  a  standard  book  of  accounts  to  maintain  records  of 
operating  costs,  staff  time  and  costs,  and  units  of  activities  for  each  cost  center. 


least  once  during  the  two  study  years  or  resided  in  the  catchment  area 
of  an  Army  MTF  during  that  time.  We  identified  this  population 
based  on  information  available  in  the  service  utilization  data  for  MTF 
care  or  network  provider  care  and  in  records  of  prescription  medica¬ 
tions  filled  by  either  MTF  pharmacies  or  the  NMOP  program.  A 
patient  was  identified  as  a  diabetic  patient  if  at  least  one  record  of  any 
of  the  following  types  was  found  for  the  time  period  between  January 
1999  and  March  2001: 


•  MTF  encounter  record  with  an  International  Classification  of 
Diseases,  Ninth  Revision  (ICD-9)  diagnosis  code  for  diabetes 
(code  250.xx)  in  any  of  the  diagnosis  code  positions  and  care 
provided  in  an  Army  facility  (all  Army  facility  IDs  including 
MTFs,  clinics,  and  TMCs); 
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•  network  provider  claim  with  an  ICD-9  diagnosis  code  for  diabe¬ 
tes  (code  250.xx)  in  any  of  the  diagnosis  code  positions  and 
patient  resided  in  the  catchment  area  of  an  Army  MTF  or  health 
center  (parent  facility  IDs  were  used  to  pick  up  those  residing  in 
the  parent  facility  catchment  area,  even  if  they  used  a  freestand¬ 
ing  clinic  with  a  separate  ID); 

•  at  least  one  prescription  filled  at  an  Army  MTF  pharmacy  that 
was  on  a  defined  list  of  diabetes  medications  (see  Appendix  A) ; 
or 

•  at  least  one  prescription  filled  through  the  NMOP  Program  that 
was  on  a  defined  list  of  diabetes  medications  (see  Appendix  A) 
and  the  patient  Zip  code  of  residence  was  located  within  the 
catchment  area  of  one  of  the  Army  MTFs  or  health  centers. 


We  identified  a  total  of  219,591  diabetic  patients  who  were 
served  by  Army  MTFs  during  the  first  year  of  this  study  and  232,123 
patients  who  were  served  during  the  second  study  year.  Additional 
details  on  the  composition  and  demographics  of  these  populations  are 
provided  in  Chapter  Three. 

The  sample  sizes  at  the  demonstration  and  control  MTFs  are 
presented  in  Table  2.3.  For  study  year  one,  a  total  of  9,820  diabetic 
patients  used  one  of  the  demonstration  sites  and  8,815  patients  used 
one  of  the  control  sites  at  least  once.  Our  study  sample  consisted  of 
patients  who  were  enrolled  at  one  of  the  MTFs  for  the  entire  year. 
For  the  first  year,  the  sample  included  6,024  patients  enrolled  at  one 
of  the  demonstration  MTFs  and  2,755  enrolled  at  one  of  the  control 
MTFs.  For  the  second  year,  the  sample  included  6,654  enrolled 
patients  at  the  demonstration  sites  and  2,841  patients  at  the  control 
sites  (because  individual  patients  were  not  followed,  no  attempt  was 
made  to  determine  degree  of  overlap  in  the  samples). 

Outcome  Measures 

The  definitions  used  to  calculate  outcome  measures  are  listed  in 
Table  2.4.  The  first  three  definitions  pertain  to  the  first  outcome 
measure  listed  in  Table  2.1. 
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Table  2.3 

Diabetic  Patient  Sample  Sizes  for  the  Demonstration  and  Control  MTFs, 
by  Study  Year 


Study  Year  One 

Study  Year  Two 

Enrollee  Group 

Demonstration 

Sites 

Control 

Sites 

Demonstration 

Sites 

Control 

Sites 

Total  MTF  users 

9,820 

8,815 

10,552 

8,683 

Enrolled  full  year 
(study  sample) 

6,024 

2,755 

6,654 

2,841 

Other  users 

3,796 

6,060 

3,898 

5,842 

NOTE:  "Enrolled  full 

year"  signifies  beneficiaries  who 

were  enrolled 

in  TRICARE 

Prime  at  this  MTF  for  all  of  their  inpatient  and  outpatient  encounters  in  the  SIDR 
and  SADR  records. 

In  addition  to  measuring  performance  on  these  indicators,  we 
performed  preliminary  tests  to  assess  the  usefulness  of  the  administra¬ 
tive  data  for  measuring  performance  on  several  additional  indicators 
for  quality  improvement  monitoring.  Our  goal  was  to  gain  a  better 
understanding  of  the  completeness,  validity,  and  reliability  of  the  data 
for  these  measures.  We  examined  the  completeness  of  the  first  set  of 
clinical  laboratory  records  that  DoD  extracted  from  the  CHCSs  of  all 
of  its  MTFs  in  2001.  We  intended  to  use  these  records  to  measure 
performance  of  HbAjC  tests  and  lipid  profiles.  We  then  assessed  the 
reliability  and  validity  of  recorded  diagnoses,  by  first  examining  the 
consistency  of  diagnostic  codes  for  each  patient  in  the  sample  and 
second  by  comparing  the  coding  of  diabetes  type  (Type  1  and  Type 
2)  on  encounter  records  with  pharmaceutical  records  of  patients’  use 
of  insulin  to  manage  their  diabetes.' 

Clinical  Laboratory  Records.  The  clinical  laboratory  data  we 
obtained  from  PASBA  were  extracted  from  the  Army  MTF  CHCSs 
for  our  study  sample  for  June  2000  through  June  2001.  We  report 
the  results  of  our  assessment  of  the  clinical  laboratory  records  in 
Table  2.5.  The  apparent  increase  in  the  number  of  laboratory 


*  The  assumption  was  that  only  patients  with  diagnostic  codes  for  Type  1  diabetes  should 
have  received  prescription  insulin  and  only  patients  with  diagnostic  codes  for  Type  2  diabe¬ 
tes  should  have  received  prescriptions  for  oral  agents.  Although  Type  2  diabetics  are  some¬ 
times  treated  with  insulin,  the  extent  of  this  practice  is  not  known. 
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Table  2.4 

Indicators  Used  to  Measure  Effects  on  Service  Utilization  Related  to 
Implementation  of  the  DoD/VA  Diabetes  Practice  Guideline 


Calculation  of  the  Indicator 


Indicator  Numerator  Denominator 


Number  of  primary  care 
visits  per  diabetic  patient 

Number  of  primary  care 
visits  per  diabetic  patient 
on  insulin 

Number  of  primary  care 
visits  per  diabetic  patient 
not  on  insulin 

Percentage  of  non-insulin 
dependent  diabetic 
patients  on  medications 
to  control  hyperglycemia 

Percentage  of  diabetic 
patients  who  had  at  least 
one  eye  exam  annually 

Number  of  ER  visits  per 
diabetic  patient 

Number  of  hospital  inpa¬ 
tient  stays  per  diabetic 
patient 


Number  of  MTF  primary 
care  visits  for  patients  in 
the  denominator 
Number  of  MTF  primary 
care  visits  for  patients  in 
the  denominator 

Number  of  MTF  primary 
care  visits  for  patients  in 
the  denominator 

Number  of  patients  in  the 
denominator  with  one 
control  medication  pre¬ 
scription®  in  the  study 
year 

Number  of  patients  in  the 
denominator  with  at 
least  one  eye  exam  dur¬ 
ing  the  study  year 
Number  of  MTF  ER  visits 
for  patients  in  the 
denominator 
Number  of  hospital  stays 
at  MTFs  or  network  hos¬ 
pitals  for  patients  in  the 
denominator 


Number  of  patients  with 
diabetes  ICD-9  code  in 
the  study  year 
Number  of  diabetes 
patients  in  the  study 
year  who  had  insulin 
prescriptions 
Number  of  diabetic 
patients  in  the  study 
year  without  insulin 
prescriptions 
Number  of  diabetic 
patients  in  the  study 
year  without  insulin 
prescriptions 

Number  of  patients  with 
diabetes  ICD-9  code  in 
the  study  year 

Number  of  patients  with 
diabetes  ICD-9  code  in 
the  study  year 
Number  of  patients  with 
diabetes  ICD-9  code  in 
the  study  year 


®  The  classes  of  diabetes  control  medications  included  are  sulfonylureas,  biguanides, 
thiazolidinediones,  and  meglitinides. 


encounters  over  the  year  suggests  that  the  data  for  early  in  the  study 
period  were  highly  incomplete.  Therefore,  we  could  not  use  these 
data  for  estimating  numbers  of  HhAjC  or  lipid  measures. 

Coding  Diabetes  Type.  Using  five-digit  ICD-9  codes,  we  tabu¬ 
lated  the  relative  frequency  of  coding  for  Type  1  and  Type  2  diabetes 
on  all  encounter  records  (including  outpatient  and  inpatient  services 
by  MTFs  and  network  providers)  for  the  patients  in  the  first  year 
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Table  2.5 

Counts  of  Clinical  Laboratory  Records  Extracted  by  DoD 
from  the  MTF  CHCS  Data,  by  Month  and  Year 


Month  and  Year 

Frequency 

Percentage 
of  Total 

June  2000 

1,511 

1.9 

July  2000 

1,697 

2.1 

August  2000 

2,893 

3.5 

September  2000 

3,697 

4.5 

October  2000 

3,427 

4.2 

November  2000 

5,238 

6.4 

December  2000 

4,090 

5.0 

January  2001 

7,907 

9.7 

February  2001 

8,492 

10.4 

March  2001 

12,620 

15.4 

April  2001 

13,816 

16.9 

May  2001 

14,526 

17.8 

June  2001 

1,806 

2.2 

sample.^  The  results  of  this  analysis  are  reported  in  Table  2.6.  We 
found  that  36.8  percent  of  the  patients  were  coded  as  Type  1  on 
some  encounter  records  hut  as  Type  2  on  other  records.  For  patients 
whose  diagnosis  was  coded  consistently  on  all  encounter  records,  3.0 
percent  were  coded  as  having  Type  1  diabetes  and  60.0  percent  were 
coded  as  Type  2  diabetes.  The  figure  of  3.0  percent  is  lower  than  that 
reported  by  the  CDC,  which  estimates  that  5  to  10  percent  of  dia¬ 
betics  are  Type  1  (CDC,  2001).  However,  the  prevalence  of  Type  1 
diabetes  would  be  expected  to  be  lower  than  average  in  this  popula¬ 
tion.^  Nevertheless,  some  of  the  patients  who  were  coded  inconsis¬ 
tently  might  actually  have  Type  1  diabetes.  Thus,  the  percentage 


^  If  the  record  had  a  diagnostic  code  of  250.01,  250.03,  250.13,  250.41,  250.43,  250.51, 
250.53,  250.61,  250.63,  250.71,  250.73,  250.83,  or  250.81,  the  record  was  coded  as  Type  1 
diabetes.  If  a  record  had  a  diagnostic  code  of  250.00,  250.02,  250.22,  250.40,  250.42, 
250.50,  250.52,  250.60,  250.62,  250.70,  250.72,  250.80,  or  250.82,  the  record  was  coded 
as  a  patient  with  Type  2  diabetes. 

^  Type  1  and  Type  2  diabetes  are  really  two  different  diseases.  The  onset  of  Type  1  (formerly 
known  as  insulin-dependent  diabetes  mellitus)  is  typically  childhood  or  adolescence;  the 
prevalence  of  Type  1  in  the  population  is  estimated  to  be  less  than  1  percent;  and  Type  1 
diabetes  is  a  disqualification  for  military  service.  The  onset  of  Type  2  (formerly  known  as 
non-insulin  dependent  diabetes  mellitus)  typically  occurs  in  the  early  40s  or  later,  although 
that  age  is  decreasing,  and  the  prevalence  of  Type  2  is  approximately  6  percent. 
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would  increase  if  those  patients  could  be  identified  accurately  by  the 
data,  and  our  analysis  confirmed  that  coding  of  diabetes  type  on 
encounter  data  does  not  provide  reliable  information. 

Matching  of  Diabetes  Type  and  Insulin  Use.  Based  on  our 
assumption  that  all  Type  1  diabetic  patients  would  use  insulin  and 
that  no  (or  very  few)  Type  2  patients  would  use  it,  we  checked  use  of 
insulin  by  reported  diabetes  type  in  the  MTF  and  NMOP  databases 
and  found  that  the  data  were  inconsistent,  as  also  shown  in  Table  2.6. 
Although  we  did  not  have  data  from  retail  pharmacies,  we  estimated 
that  they  accounted  for  only  a  small  fraction  of  insulin  prescriptions. 
When  we  matched  insulin  use  to  diabetes  type,  as  derived  from  the 
ICD-9  codes  on  service  use  encounters,  we  found  that  only  20.7  per¬ 
cent  of  the  patients  identified  consistently  as  Type  1  diabetics  had 
pharmacy  records  for  insulin  prescriptions,  and  27.7percent  of  those 
with  both  Type  1  and  Type  2  codes  had  insulin  prescriptions. 

These  results  reveal  two  separate  issues  in  the  administrative  data 
that  merit  attention  by  MEDCOM.  First,  the  very  low  percentage  of 
Type  1  diabetic  patients  with  insulin  prescriptions  indicates  that  the 
pharmaceutical  data  we  were  provided  were  incomplete.  This  issue 
raises  a  question  about  the  accuracy  of  the  data  used  to  test  trends  for 
some  of  the  indicators  in  this  study,  and  it  also  poses  a  challenge 
regarding  the  validity  of  DoD  administrative  data  for  ongoing  moni¬ 
toring  of  diabetes  management  practices.  Second,  the  percentage  of 
inconsistently  coded  patients  with  insulin  prescriptions  suggests  that 


Table  2.6 

Comparison  of  Diabetes  Type  Reported  in  Diagnostic  Codes  and 
Use  of  Insulin,  Study  Year  One 


Estimated  Diabetes  Type 

Unknown 

All  Type  1 
codes 

All  Type  2 
codes 

Both  Types 

1  and  2 

Total 

No  insulin  prescription 

Number  of  patients 

19 

207 

5,084 

2,334 

7,644 

Percentage 

100.0 

79.3 

96.6 

72.3 

87.2 

Prescription  for  insulin 

Number  of  patients 

0 

54 

179 

894 

1,127 

Percentage 

0.0 

20.7 

3.4 

27.7 

12.9 

Total  number  of  patients 

19 

261 

5,263 

3,228 

8,771 
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this  group  includes  some  Type  1  diabetics.  These  findings  revealed 
the  need  to  determine  how  best  to  achieve  accurately  automated 
records  of  diabetes  types  for  ongoing  monitoring  of  care  and  other 
management  or  quality  improvement  purposes. 

Definitions  of  Other  Key  Variables 

Variables  for  enrollment  status,  health-care  service  use,  and  medica¬ 
tions  were  derived  for  calculation  of  the  indicators  being  analyzed. 
We  also  defined  variables  for  the  gender,  age,  and  insulin  use  status  of 
each  patient  in  our  study  sample.  These  variables  are  summarized 
here,  and  additional  coding  details  are  provided  in  Appendix  A. 

Enrollment  Type.  Two  variables  were  derived  and  used  to  define 
enrollment  status  at  a  demonstration  or  control  MTF.  The  first  was  a 
variable  that  identified  whether  a  patient  was  ever  enrolled  at  one  of 
these  MTFs  or  enrolled  with  a  network  provider  but  resided  in  the 
catchment  area  of  one  of  the  MTFs.  The  second  variable  identified 
whether  the  patient  was  continuously  enrolled  in  the  same  place  or 
changed  enrollment  during  the  relevant  year.  The  enrollment  type 
variables  were  also  used  to  identify  comparison  populations  for  some 
of  the  analyses. 

Patient  Age.  The  variable  for  patient  age  was  defined  as  the  age 
of  each  patient  at  the  end  of  each  calendar  year,  which  was  calculated 
using  the  date  of  birth  variable  established  on  the  master  file.  We  also 
derived  a  categorical  variable  for  patient  age,  which  was  used  in  some 
of  the  analyses.  The  age  categories  were  less  than  18  years,  18-44 
years,  45-64  years,  and  greater  than  65  years. 

Insulin  Use.  A  diabetic  patient  was  identified  as  an  insulin  user  if 
he  or  she  had  two  or  more  insulin  prescriptions  in  a  study  year.  Insu¬ 
lin  prescriptions  included  those  for  Humalog,  Humulin,  Iletin,  Iso- 
phane,  Insulatard,  Lantis,  Mixtard,  Novolin,  Relion,  human  insulin, 
beef  insulin,  pork  insulin.  Protamine  zinc,  or  Velosulin. 

Use  of  Primary  Care  Services.  For  each  patient  in  the  study  sam¬ 
ple,  primary  care  visits  were  identified  from  the  data  in  the  SADR, 
which  contains  data  for  MTF  visits.  All  visits  for  MTF  family  prac¬ 
tice,  internal  medicine,  pediatric,  adolescent,  other  primary  care,  or 
flight  medicine  clinics  were  coded  as  primary  care  visits.  A  variable 
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was  created  with  the  numher  of  primary  care  visits  for  each  patient, 
and  primary  care  visit  rates  were  calculated  as  the  sum  of  visits  across 
all  patients  in  a  group  divided  hy  the  numher  of  these  patients. 

Use  of  Noninsulin  Drugs  for  Glycemic  Control.  Any  patient  with 
one  or  more  prescriptions  for  noninsulin  glycemic  control  agents  was 
defined  as  a  person  using  noninsulin  drugs  to  control  HhAjC  levels.  A 
prescription  was  defined  as  a  noninsulin  glycemic  control  drug  if  it 
was  for  one  of  the  sulfonylureas,  higuanides,  thiazolidinediones,  or 
meglitinides. 

Eye  Exams.  Extender  codes  have  heen  established  in  the  ADS  to 

record  eye  exams  and  foot  exams  for  diabetic  patients  (V72.9 _ 3  for 

foot  exam  and  V72.9 _ 4  for  eye  exam).  However,  these  codes 

became  available  only  after  the  demonstration,  so  none  of  the  MTFs 
in  this  study  could  use  them  to  document  the  exams.  To  determine 
numbers  of  eye  exams,  we  identified  visits  at  optometry  or  ophthal¬ 
mology  clinics,  and  any  patient  with  at  least  one  of  these  visits  in  a 
study  year  was  coded  as  having  an  annual  eye  exam.  To  the  extent 
that  the  new  codes  are  used  by  the  Army  MTFs,  these  measures 
would  be  tracked  in  the  future. 

Use  of  ER  Services.  We  were  able  to  measure  ER  visits  only  for 
MTF  ERs  because  we  did  not  have  the  data  needed  to  identify  ER 
visits  in  the  network  provider  outpatient  data."*  The  ER  visits  proba¬ 
bly  are  undercounted  because  of  this  missing  data,  but  we  believe  that 
the  undercount  is  small  because  most  patients  enrolled  at  an  MTF  are 
likely  to  use  the  MTF  ER  when  they  need  such  care.  ER  visit  rates 
were  calculated  as  the  sum  of  visits  across  all  patients  in  a  group 
divided  by  the  number  of  patients. 

Hospital  Inpatient  Use.  A  variable  was  created  for  hospital  inpa¬ 
tient  stays,  which  included  stays  at  MTFs  and  community  hospitals 
that  are  network  providers.  Hospitalization  rates  were  calculated  as 
the  sum  of  all  inpatient  stays  across  the  patients  in  the  sample  divided 


^  There  is  a  variable  in  the  HCSRs  that  identifies  the  type  of  outpatient  encounter,  so  it  is 
possible  to  identify  network  provider  ER  visits.  However,  we  had  not  obtained  that  variable 
in  the  network  provider  data  for  this  study,  so  we  had  to  work  with  only  the  MTF  ER 
information. 
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by  the  number  of  patients.  This  variable  was  used  to  create  a  categori¬ 
cal  variable  (zero,  one,  or  two  or  more  stays)  that  was  the  dependent 
variable  for  the  ordered  logistic  analysis  to  test  trends  in  hospitaliza¬ 
tion  rates. 

Data  Collection 

An  interrupted  time  series  control-group  design  was  used.  To  allow 
tracking  of  time  trends,  data  were  collected  quarterly.  Trends  for  the 
baseline  measures  were  estimated  for  one  year  before  implementation, 
and  trends  for  the  implementation  outcomes  were  estimated  for  one 
year  after  implementation.  This  approach  allowed  us  to  estimate 
annual  utilization  measures  and  also  controlled  for  any  seasonality 
effects  on  diabetes  care.  Data  from  the  control  group  of  five  addi¬ 
tional  MTFs  were  used  to  adjust  for  underlying  historical  trends  that 
might  confound  the  validity  of  findings  at  the  demonstration  MTFs. 

Analytic  Methods 

The  first  step  in  the  analysis  was  to  calculate  values  for  each  indicator 
in  each  quarter-year  of  the  study  period.  For  each  measure,  we  then 
estimated  the  baseline  performance  for  the  MTFs,  described  quarterly 
trends  for  the  demonstration  and  control  sites,  and  tested  the  statisti¬ 
cal  significance  of  any  observed  differences  in  performance  of  the 
demonstration  site  compared  to  the  control  sites.  See  Appendixes  A 
and  C  for  details  on  the  statistical  tests. 

Baseline  Benchmarking.  Data  from  study  year  one  were  used  as 
baseline  measures  for  the  five  indicators.  For  each  indicator,  we  com¬ 
pared  the  performance  of  each  MTF  to  the  mean  performance  of  all 
other  MTFs  combined  (i.e.,  excluding  the  index  MTF).  The  baseline 
performance  information  for  MTFs  is  reported  in  Chapter  Three.  We 
did  not  adjust  for  multiple  comparisons,  which  can  increase  prob¬ 
ability  of  Type  1  errors  (false  negatives).  Although  significance  levels 
are  reported  for  differences  from  the  mean,  the  clinical  significance  of 
observed  differences  among  MTFs  needs  to  be  considered  when  ana¬ 
lyzing  results,  particularly  for  indicators  where  no  guideline  exists. 

Descriptions  of  Trends  for  Indicators.  To  describe  trend  infor¬ 
mation,  we  prepared  tables  and  graphs  displaying  estimates  for  the 
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indicators  over  the  two  study  years,  aggregated  separately  for  the 
demonstration  and  control  sites.  In  some  cases,  we  found  substantial 
differences  in  performance  levels  or  trends  between  the  demonstra¬ 
tion  sites  or  among  the  control  sites.  We  examined  the  effects  of  these 
differences  on  overall  trends  by  describing  trends  separately  for  each 
demonstration  site  or  by  describing  aggregate  trends  for  the  relevant 
group  of  sites  after  excluding  an  MTF  with  outlying  values.  The 
quantitative  results  were  compared  to  the  implementation  strategies 
of  the  demonstration  sites  to  better  interpret  the  observed  trends. 
This  step  allowed  us  to  assess  the  extent  to  which  those  strategies  were 
reflected  in  observed  service  changes.  These  results  are  reported  in 
Chapter  Five. 

Testing  the  Significance  of  Indicator  Trends.  The  final  step  of 
the  analysis  was  to  test  whether  observed  changes  in  service  rates  or 
medication  use,  if  any,  were  large  enough  to  be  statistically  signifi¬ 
cant,  after  controlling  for  temporal  trends  and  for  patient  characteris¬ 
tics.  For  each  indicator,  we  estimated  a  regression  model  with  the 
dependent  variable  being  the  indicator  of  interest  and  the  predictor 
variables,  including  a  dichotomous  variable  for  demonstration  or  con¬ 
trol  site,  a  set  of  dummy  variables  for  the  quarter-year  periods,  and 
variables  for  the  patient  characteristics. 

To  test  for  changes  in  outcomes  during  study  year  two  at  the 
demonstration  sites,  we  also  included  one  or  more  interaction  terms 
for  demonstration  sites  and  each  quarter  of  the  intervention  period. 
To  determine  the  final  specification  of  the  interaction  terms,  we  were 
guided  by  the  observed  trends  in  the  measures  and  the  significance  of 
the  coefficients  on  the  interaction  term  for  each  quarter.  The  results 
of  the  analysis  are  presented  in  Chapter  Five,  and  the  specification  of 
each  model  and  detailed  results  of  the  modeling  are  reported  in 
Appendix  B. 


Estimating  the  Costs  of  Care 

The  analysis  of  costs  of  care  had  two  purposes:  to  gain  an  under¬ 
standing  of  the  costs  MTFs  incur  for  health  care  services  for  diabetic 
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patients  and  to  evaluate  whether  introduction  of  the  diabetes  practice 
guideline  affected  those  costs.  The  sample  used  for  the  analysis  was 
populated  hy  the  patients  served  hy  the  demonstration  MTFs.  To 
estimate  the  costs  of  care,  MEPRS  financial  data  were  used  to  develop 
sets  of  unit  costs  for  different  types  of  inpatient  and  outpatient 
encounters.  The  relevant  estimated  unit  cost  was  then  applied  to  each 
unit  of  service  in  the  SIDR  and  SADR  encounter  records  (where  each 
record  represented  one  unit  of  service). 

In  developing  our  unit  cost  estimation  methodology,  our  goal 
was  to  derive  cost  estimates  that  captured  all  MTF  costs  of  care  for 
inpatient  or  outpatient  events  and  were  sensitive  to  variations  in  the 
intensity  of  resources  required  to  provide  health  care  of  different 
types.  This  cost  estimation  method  was  developed  originally  as  part  of 
the  RAND  evaluation  of  the  Medicare-DoD  Subvention  Demonstra¬ 
tion.  Appendix  A  shows  the  details  of  the  calculation  of  the  unit  costs 
and  their  application  to  each  MTF  inpatient  and  outpatient  encoun¬ 
ter  record. 

We  designed  the  cost  estimation  methodology  with  technical 
consultation  from  SRA  International,  the  TRICARE  Management 
Activity  (TMA)  contractor  that  developed  the  Patient-Fevel  Cost 
Allocation  (PFCA)  method  used  to  design  the  financial  provisions  of 
the  demonstration.  The  methodology  we  developed  is  an  adaptation 
of  the  approach  SRA  took  for  developing  the  PFCA  method. 

For  this  cost  analysis,  we  used  cost  and  workload  data  for  fiscal 
year  (FY)  1998  that  SRA  generated  for  MTF  outpatient  clinics  or 
inpatient  wards  for  all  Army  MTFs  included  in  this  evaluation.  The 
estimated  unit  costs  included  total  direct  and  indirect  expenses  for 
each  MTF  cost  center  (ward  or  clinic),  including  direct  expenses  for 
staff  time  and  supplies  as  well  as  indirect  expenses  for  ancillary  clinical 
services,  administrative  services,  and  maintenance  and  other  support 
services. 

We  updated  the  unit  costs  to  FY  1999  estimates  by  applying  an 
inflation  factor  of  1.4  percent.  These  same  unit  costs  were  applied  to 
encounters  for  both  study  years.  By  holding  costs  constant  over  time, 
any  observed  changes  in  costs  between  study  years  one  and  two  can 
be  attributed  to  changes  in  utilization.  We  tested  two  references  for 
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Medicare  cost  increases  to  determine  the  1.4  percent  inflation  rate 
(details  in  Appendix  A). 

Different  unit  costs  were  applied  to  each  MTF  encounter  for 
different  types  of  hospital  inpatient  stays  (e.g.,  medical,  surgical)  or 
outpatient  visits  to  different  types  of  clinics.  For  a  small  percentage  of 
clinics  for  which  data  were  not  sufficient  to  calculate  clinic-specific 
cost  estimates,  we  applied  an  MTF-average  unit  cost  for  outpatient 
services.  Then  we  aggregated  all  encounters  for  each  diabetic  patient 
in  the  study  to  the  patient  level  and  used  these  aggregate  costs  to 
analyze  per  capita  costs.  Costs  also  were  analyzed  at  the  encounter 
level  to  assess  the  distribution  of  MTF  costs  between  TRICARE 
Prime  enrollees  and  other  beneficiaries  and  to  assess  the  extent  to 
which  costs  are  distributed  between  MTFs  and  network  providers  in 
the  community.  From  the  patient  perspective,  we  examined  the  total, 
inpatient,  and  outpatient  costs  of  care  per  patient  for  MTF  services, 
looking  separately  at  MTF  enrollees  and  nonenrollees. 


CHAPTER  THREE 


Diabetic  Population  and  Practices  at  the  Baseline 


Because  an  understanding  of  the  patient  population  and  baseline  per¬ 
formance  of  key  care  processes  can  influence  the  strategies  chosen  for 
implementation  of  care  guidelines,  we  precede  our  report  of  the 
evaluation  of  the  implementation  strategy  and  outcomes  with  a 
description  of  the  population  served  by  the  MTFs  and  the  results  of 
an  analysis  of  baseline  care  processes. 

Diabetic  patients  need  ongoing  care  management.  Therefore, 
TRICARE  beneficiaries  with  diabetes  are  regular  users  of  services 
covered  by  their  TRICARE  benefits.  These  services  may  be  provided 
by  MTEs  or  civilian  network  providers,  depending  on  the  patients’ 
TRICARE  enrollment  status,  proximity  to  an  MTE,  and  the  avail¬ 
ability  of  needed  services  at  the  MTE.  The  primary  patient  popula¬ 
tion  an  MTE  serves  consists  of  TRICARE  Prime  enrollees  for  whom 
that  MTE  is  their  primary  care  manager.  MTEs  also  serve  patients 
who  are  not  enrolled  in  TRICARE  Prime  or  whose  primary  care 
manager  is  at  another  location  but  who  need  care  while  in  the  area  or 
are  referred  from  another  MTE  or  network  provider. 


The  Diabetic  Population  Served  by  Army  MTFs 

The  Army  MTEs  served  close  to  230,000  diabetic  patients  during  the 
first  (baseline)  year  of  our  study.  Table  3.1  shows  the  percentages  of 
diabetic  patients  identified  by  using  each  of  the  encounter  data 
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Table  3.1 

Identification  of  the  Diabetic  Population  Served  by  Army  MTFs, 
by  Study  Year 


SIDR/SADR 

PEC 

Pharmacy 

Network 

Provider 

NMOP 

Total 

study  year  one 

Army  active-duty 

1,745 

654 

12 

a 

2,411 

Army  family  member 

12,106 

21,323 

1,920 

a 

8,179 

Army  retired 

11,561 

20,562 

2,553 

a 

35,349 

Retired  family 

12,569 

118,567 

7,840 

a 

34,676 

member 

Other  services 

4,479 

2,955 

745 

a 

138,976 

NMOP  only 

a 

a 

a 

9,300 

9,300 

Total  patients,  year 
one 

42,460 

164,061 

13,070 

9,300 

228,891 

Percentage  of  total 

18.5 

71.7 

5.7 

4.1 

100.0 

Study  year  two 

Army  active-duty 

1,621 

717 

28 

a 

2,366 

Army  family  member 

4,297 

3,204 

555 

a 

8,056 

Army  retired 

12,299 

22,766 

1,853 

a 

36,918 

Retired  family 

11,635 

22,206 

2,380 

a 

36,221 

member 

Other  services 

12,635 

128,777 

7,150 

a 

148,562 

NMOP  only 

a 

a 

a 

10,117 

10,117 

Total  patients,  year 
two 

42,487 

177,670 

11,966 

10,117 

242,240 

Percentage  of  total 

17.5 

73.4 

4.9 

4.2 

100.0 

®NMOP  claims  did  not  have  data  on  the  military  status  of  beneficiaries,  so  we  could 
not  classify  any  patients  who  were  identified  only  through  NMOP  claims. 


sources.  For  study  year  one,  18.5  percent  of  the  patients  were  identi¬ 
fied  using  the  SIDR  and  SADR  data,  and  another  71.7  percent  (for 
whom  there  were  no  SIDR  or  SADR  records)  were  identified  through 
the  PEC  pharmacy  data.  The  network  provider  data  added  only 
another  5.7  percent,  and  the  NMOP  data  added  the  remaining  4.1 
percent.  Similar  percentages  were  found  for  study  year  two.  More 
than  half  the  diabetic  patients  were  active-duty  personnel,  retirees,  or 
family  members  of  services  other  than  the  Army.  Among  those  affili¬ 
ated  with  the  Army,  all  but  a  small  fraction  were  either  retired  Army 
personnel  or  family  members,  and,  given  the  nature  of  the  diseases,  it 
is  not  surprising  that  only  a  small  proportion  of  diabetics  were  active- 
duty  Army  personnel  (2,41 1  in  study  year  one;  see  Chapter  Two). 

The  demographic  characteristics  of  the  diabetic  patients, 
reported  in  Table  3.2,  reflect  their  military  status.  Overall,  42.8  per- 
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cent  of  the  patients  in  study  year  one  were  45  to  64  years  of  age  and 
46.2  percent  were  65  years  of  age  or  older.  The  percentages  were 
similar  for  study  year  two.  These  percentages  closely  match  those  for 
retired  personnel  and  family  members,  who  constitute  the  majority  of 
diabetic  TRICARE  patients.  The  active-duty  personnel  and  family 
members  with  diabetes  have  a  younger  age  distribution. 


Enrollment  Status  and  Use  of  MTF  Services 


The  enrollment  status  of  diabetic  patients  seen  at  MTFs  was  also 
examined,  for  two  reasons.  First,  MTFs  should  be  able  to  manage 


Table  3.2 

Demographic  Characteristics  of  the  Diabetic  Population  Served  by  Army 
MTFs,  by  Study  Year 


Army 

Army 

Retired 

All 

Active- 

Family 

Army 

Family 

Other 

NMOP 

Patients 

Duty 

Member 

Retired 

Member 

Services 

Only 

Study  year  one 
Distribution  by  age 
Total 

228,883 

2,411 

8,176 

35,349 

34,676 

138,971 

9,300 

Less  than  18 

years®  (%) 

1.4 

0.2 

11.1 

0.0 

1.1 

1.3 

0.6 

18-44  years  (%) 

9.7 

66.3 

53.4 

2.6 

5.5 

9.4 

3.7 

45-64  years  (%) 

42.8 

30.2 

21.9 

42.9 

46.3 

43.1 

45.1 

65+  years  (%) 
Distribution  by  gen¬ 

46.2 

3.3 

13.6 

54.5 

47.1 

46.2 

50.7 

der 

Male  (%) 

50.2 

77.1 

12.1 

98.7 

1.9 

51.6 

51.5 

Study  year  two 
Distribution  by  age 
Total 

242,239 

2,366 

8,056 

36,918 

36,221 

148,561 

10,117 

Less  than  18 

years®  (%) 

1.2 

0.2 

9.1 

0.0 

0.9 

1.1 

0.6 

18-44  years  (%) 

9.4 

65.0 

53.9 

2.3 

5.0 

9.3 

3.0 

45-64  years  (%) 

41.6 

31.0 

22.6 

42.5 

45.1 

41.7 

41.5 

65+  years  (%) 
Distribution  by  gen¬ 

47.9 

3.8 

14.3 

55.2 

48.9 

47.9 

54.8 

der 

Male  (%) 

50.1 

76.2 

11.1 

98.7 

1.9 

51.5 

50.1 

'“The  finding  of  active-duty  personnel  under  age  18  may  represent  an  error  in  the 
records. 
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care  more  effectively  for  their  own  enrollees  than  for  intermittent 
users  who  also  may  he  obtaining  care  from  local  network  providers  or 
other  MTFs.  For  this  reason,  the  MTFs  should  he  held  accountable 
first  for  care  of  their  own  enrollees,  and  any  performance-monitoring 
system  should  use  measures  calculated  specifically  for  this  population. 
Second,  for  accurate  calculation  of  performance  measures,  all  care 
obtained  by  a  patient  population  of  interest  should  be  taken  into 
account.  For  example,  hospitalization  rates  would  be  undercounted  if 
hospital  stays  in  community  (nonmilitary)  hospitals  were  excluded 
from  the  counts  used  to  calculate  these  rates. 

Our  analysis  revealed  the  importance  of  using  both  MTF 
service  encounter  data  and  network  provider  claims  to  gain  a  full 
understanding  of  service  use  patterns.  As  shown  in  Table  3.3,  61.8 
percent  of  MTF  outpatient  or  ER  visits  were  provided  for  members 
of  the  diabetic  patient  population  who  were  enrolled  in  TRICARE 
Prime  at  those  MTEs.  Another  37.6  percent  of  the  visits  were  for 
patients  not  enrolled  at  the  MTE,'  and  fewer  than  1  percent  of  ser¬ 
vices  were  for  patients  enrolled  with  network  providers.  In  contrast, 
virtually  all  inpatient  care  provided  by  MTEs  was  for  their  own 
enrollees. 

Network  providers  had  a  different  pattern  of  service  provision. 
An  estimated  67.9  percent  of  network  outpatient  and  ER  visits  were 
provided  to  patients  enrolled  in  TRICARE  Prime  with  network  pro¬ 
viders.  Another  15.8  percent  were  for  patients  enrolled  at  MTEs,  and 
16.2  percent  were  for  nonenrollees.  A  different  distribution  was 
found  for  network  provider  inpatient  services,  with  1 1.4  percent  pro¬ 
vided  to  MTE  enrollees,  43.7  percent  to  network  provider  enrollees, 
and  another  44.9  percent  provided  to  nonenrollees. 

These  data  are  also  shown  graphically  in  Appendix  C.  The  data 
show  that  services  provided  to  nonenrollees  represent  a  substantial 


'  The  population  of  nonenrolled  patients  includes  a  large  percentage  of  individuals  over  65 
years  of  age  who  are  eligible  for  Medicare  as  well  as  DoD  health  benefits.  Because  Medicare 
is  their  primary  payer,  these  patients  have  access  to  MTF  services  only  on  a  space-available 
basis. 
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Table  3.3 

Enrollment  Status  for  Patients  Receiving  Diabetes  Care  at  Army  MTFs  or 
Network  Providers,  by  Study  Year 


Diabetes  Outpatient/ 

ER  Visits  Diabetes  Admissions 


Number  of  Percentage  Number  of  Percentage 
Encounters  of  Total  Encounters  of  Total 


Study  year  one 
Service  at  MTFs 


Prime  enrolled  at  the  MTF 

102,709 

61.8 

8,451 

98.9 

Prime  enrolled  with  a  network 
provider 

1,011 

0.6 

90 

1.1 

Not  enrolled 

62,419 

37.6 

1 

0.0 

All  patients 

166,139 

100.0 

8,542 

100.0 

Service  at  network  providers 
Prime  enrolled  at  an  MTF 

17,309 

15.8 

549 

11.4 

Prime  enrolled  in  network 

74,278 

67.9 

2,111 

43.7 

Not  enrolled  in  Prime 

17,739 

16.2 

2,166 

44.9 

All  patients 

109,326 

100.0 

4,826 

100.0 

Study  year  two 

Service  at  MTFs 

Prime  enrolled  at  the  MTF 

125,446 

67.2 

8,405 

98.5 

Prime  enrolled  with  a  network 
provider 

1,033 

0.6 

125 

1.5 

Not  enrolled  in  Prime 

60,174 

32.2 

5 

0.1 

All  patients 

186,653 

100.0 

8,535 

100.0 

Network  provider  service 

Prime  enrolled  at  an  MTF 

18,865 

19.9 

690 

14.1 

Prime  enrolled  in  network 

61,971 

65.3 

2,194 

44.7 

Not  enrolled  in  prime 

14,033 

14.8 

2,019 

41.2 

All  patients 

94,869 

100.0 

4,903 

100.0 

share  of  MTF  health  care  resources.  The  data  also  show  that  MTF 
enrollees  use  a  mix  of  MTF  and  network  provider  services,  so  all 
services  must  he  counted  to  obtain  accurate  estimates  of  health-care 
use  rates. 


Baseline  Performance  on  Diabetes  Care  Measures 


The  number  of  diabetic  patients  who  used  each  of  the  demonstration 
and  control  sites  is  reported  in  Table  3.4.  These  data  include  separate 
counts  of  all  patients  who  used  each  MTF  at  least  once  during  each 
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Table  3.4 

Diabetic  Population  Using  the  Demonstration  and  Control  MTFs, 
by  Study  Year 


Study  Year  One 

Study  Year  Two 

study  Site 

Any  Use 

Enrolled 
Full  Year 

Any  Use 

Enrolled 
Full  Year 

Demonstration  sites 

Demo  1 

3,303 

2,178 

3,635 

2,428 

Demo  2 

258 

111 

249 

128 

Demo  3 

4,132 

2,149 

4,039 

2,216 

Demo  4 

899 

680 

1,031 

825 

Demo  5 

1,228 

906 

1,598 

1,057 

All  Demos 

9,820 

6,024 

10,552 

6,654 

Control  sites 

Control  1 

3,021 

1,339 

3,056 

1,429 

Control  2 

695 

159 

625 

96 

Control  3 

3,585 

351 

3,643 

367 

Control  4 

867 

489 

864 

591 

Control  5 

647 

417 

495 

358 

All  Controls 

8,815 

2,755 

8,683 

2,841 

NOTE:  "Enrolled  full  year"  signifies  beneficiaries  who  were  enrolled 
in  TRICARE  Prime  at  this  MTF  for  all  of  their  inpatient  and  out¬ 
patient  encounters  recorded  in  SIDR  and  SADR. 


Study  year  and  of  the  subset  of  patients  who  were  enrolled  at  the 
MTF  all  year,  as  indicated  by  enrollment  status  codes  on  the  encoun¬ 
ter  records.  Overall,  enrollees  represented  a  larger  proportion  of  total 
diabetics  seen  at  the  demonstration  sites  than  did  enrollees  at  the  con¬ 
trol  sites  (almost  60  percent  versus  less  than  35  percent).  Two  control 
sites  with  larger  numbers  of  patients  were  important  sources  of  this 
difference,  but  the  difference  was  also  found  for  smaller  control  sites. 

We  calculated  average  baseline  values  for  the  five  indicators  of 
guideline  effects  for  the  baseline  time  period — ^April  1999  through 
March  2000 — for  each  of  the  ten  MTFs  included  in  the  study.  We 
also  calculated  an  overall  average  as  a  benchmark  against  which  each 
MTF  was  compared.  These  ten  facilities  represent  approximately  one- 
quarter  of  the  Army  MTFs,  with  respect  to  patient  population  served. 

These  baseline  comparisons  are  diagnostic  tools  that  show  the 
extent  of  variation  across  facilities  in  the  provision  of  diabetes  services 
and  highlight  particular  facilities  or  aspects  of  care  that  merit  targeted 
intervention  for  strengthening  practices.  The  direction  provided  by 
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the  DoD/VA  diabetes  guideline  should  be  considered  when  inter¬ 
preting  the  baseline  performance  data  and  formulating  quality 
improvement  interventions  for  diabetes  care. 

As  shown  in  Table  3.5,  the  guideline  places  an  emphasis  on  early 
and  ongoing  care  management  for  diabetes,  including  careful  control 
of  glycemia  (as  measured  by  HbAjC  levels)  as  well  as  regular  screening 
for  signs  of  such  organ  system  complications  as  neuropathy  or  effects 
on  vision.  Thus,  well-managed  patients  should  make  regular  visits  to 
primary  care  providers  for  management  of  their  diabetes.  Similarly, 
high  percentages  of  patients  should  have  at  least  one  eye  examination 
annually  to  check  for  early  symptoms  of  vision  problems.  Wide  varia¬ 
tion  across  MTFs  on  any  given  measure  suggests  that  MTFs  may  not 
be  providing  care  consistently,  which  could  include  overtreatment  in 
some  cases  and  undertreatment  in  others. 

Distributions  of  MTFs  on  Diabetes  Measures 

Figures  3.1  through  3.5  show  the  baseline  performance  of  the  study 
MTFs  on  the  six  indicators  of  diabetes  care.  The  first  bar  on  the  left 
of  each  graph  is  the  overall  average  baseline  performance  for  all  ten 
MTFs,  and  the  remaining  bars  show  the  values  for  each  of  the  MTFs, 
numbered  to  protect  confidentiality.  Asterisks  following  the  site 
number  indicate  a  significant  difference  in  performance  from  the 
average  for  the  other  MTFs  (**  for  p  less  than  0.01). 

Table  3.5 

Rationale  for  Diabetes  Indicators  Provided  by  the  Guideline 


Indicator 


Guideline  Direction 


Primary  care  visits 


Oral  hypoglycemic  agents 
for  patients  not  using 
insulin 

Annual  eye  examinations 
ER  visits 

Hospital  inpatient  stays 


Monitor  patients'  glycemic  status  and  organ  systems 
regularly.  Low  visit  rates  may  indicate  inadequate  care 
management. 

Perform  proactive  tests  for  glycemic  control;  prescribe 
and  adjust  oral  agents  as  indicated  by  results. 

Conduct  an  eye  examination  annually,  more  frequently 
if  indicated,  to  prevent  complications. 

Ensuring  effective  care  management  should  reduce 
rates  of  use  of  ER  care. 

Ensuring  effective  care  management  should  reduce 
need  for  inpatient  care. 
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Figure  3.1 

Baseline  Annual  Primary  Care  Visits  per  100  Diabetes  Patients  for  Insulin  and 
Non-Insulin  Users,  Total  and  by  Site 


**p  =  less  than  0.01 . 


RAND  MG277-3.1 


Primary  Care  Visits.  The  average  rates  of  primary  care  visits  for 
diabetic  patients  who  use  insulin  as  well  as  patients  who  do  not  use 
insulin  reveal  substantial  inter-MTF  differences,  all  of  which  are  sta¬ 
tistically  significant  (Figure  3.1).  The  visit  rates  for  insulin  users  were 
higher  than  those  for  non— insulin  users  for  all  the  MTFs,  although 
the  differences  between  them  vary  across  MTFs.  Visit  rates  for  insulin 
users  ranged  from  fewer  than  200  visits  to  more  than  750  visits  per 
100  patients,  while  those  for  non-insulin  users  ranged  from  fewer 
than  200  to  more  than  500  per  100  patients. 

Use  of  Oral  Hypoglycemic  Agents.  The  baseline  percentages  of 
non-insulin  using  patients  on  these  agents  ranged  from  40  to  60  per¬ 
cent  for  all  the  study  sites,  except  for  one  control  site  with  a  level  of 
approximately  10  percent  (Figure  3.2).  This  figure  suggests  that  phar- 
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Figure  3.2 

Baseline  Percentages  of  Non-Insulin  Using  Diabetic  Patients  Treated  with 
Oral  Hypoglycemic  Agents,  Total  and  by  Site 


60 


**p  =  less  than  0.01 . 


RAND  MG277-3.2 

maceutical  data  reported  for  this  site  may  have  been  incomplete.  All 
differences  observed  across  the  sites  were  statistically  significant. 

Annual  Eye  Exams.  The  percentage  of  diabetic  patients  who  had 
at  least  one  eye  examination  during  the  baseline  year  varied  substan¬ 
tially  across  study  sites  (Figure  3.3).  Two  control  sites  showed  very 
low  percentages,  which  may  reflect  missing  data.  Values  for  the 
remaining  sites  ranged  from  approximately  four  percent  to  greater 
than  20  percent,  and  the  differences  across  MTFs  were  statistically 
significant.  The  DQIP  standard  for  annual  eye  exams  is  1 00  percent. 

ER  Visits  and  Hospitalization  Rates.  Substantial  and  statistically 
significant  variation  was  observed  across  MTFs  for  both  ER  visits 
(Figure  3.4)  and  hospitalization  (inpatient  stays)  (Figure  3.5).  Several 
MTFs  with  higher  rates  of  ER  visits  had  lower  rates  of  hospitalization 
while  some  with  lower  rates  of  ER  visits  had  higher  hospitaliza- 
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Figure  3.3 

Baseline  Percentage  of  Diabetic  Patients  with  at  Least  One  Eye  Examination 
Annually,  Total  and  by  Site 

25  - 


**p  =  less  than  0.01 . 


RAND  MG277-3.3 


tion  rates.  Missing  data  may  also  be  a  concern  for  these  measures 
because  two  MTFs  showed  quite  low  ER  visit  rates  and  one  MTF 
had  unexpectedly  low  rates  of  hospitalization. 


Summary 

The  baseline  data  analysis  produced  the  following  findings: 

•  The  diabetic  population  served  by  Army  MTFs  is  large,  primar¬ 
ily  consisting  of  older,  Type  2  diabetics  (retirees  from  all  services 
and  their  dependents). 

•  The  majority  of  diabetic  patients  served  by  the  MTFs  were 
TRICARE  Prime  beneficiaries  for  whom  that  MTF  was  their 
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Figure  3.4 

Baseline  Annual  ER  Visits  per  100  Diabetic  Patients,  Total  and  by  Site 

9.0  - 


8.0 


**p  =  less  than  0.01 . 


RAND  MG277-3.4 

Figure  3.5 

Baseline  Annual  Inpatient  Stays  per  100  Diabetes  Patients,  Total  and  by  Site 


35 


**p  =  less  than  0.01 . 


RAND  MG277-3.S 
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primary  care  manager.  Nevertheless,  a  substantial  proportion  of 
diabetic  patients  seen  at  the  MTFs  were  enrolled  at  other  facili¬ 
ties  (MTFs  or  civilian  network  sites).  These  patients  would  be 
expected  to  receive  at  least  some  of  their  diabetic  care  at  their 
site  of  enrollment  and  might  account  for  some  of  the  low  mea¬ 
sures  on  the  five  indicators. 

•  Significant  differences  were  observed  across  the  sites  in  the  base¬ 
line  values  for  each  of  the  five  care  indicators.  However,  the 
importance  of  these  differences  depends  on  how  the  actual  per¬ 
formance  at  each  site  varies  from  recommended  guidelines, 
where  applicable. 

•  Some  low  measures  suggested  incomplete  data. 

•  The  baseline  comparisons  can  be  used  as  benchmarks  to  detect 
differences  across  facilities  and  to  identify  areas  where  practice 
improvement  efforts  may  be  needed. 


CHAPTER  FOUR 

The  Guideline  Implementation  Process 


The  diabetes  guideline  demonstration  was  the  third  test  of  an  imple¬ 
mentation  approach  that  coordinated  actions  at  the  corporate 
(MEDCOM)  and  local  (MTF)  levels  to  achieve  best  practices. 
MEDCOM  defined  the  desired  clinical  practices  (as  specified  in  the 
DoD/VA  practice  guideline)  and  key  metrics  to  measure  attainment 
of  those  practices,  provided  tools  to  assist  the  MTFs  as  they  intro¬ 
duced  new  practices  in  response  to  the  guideline,  and  facilitated  MTF 
guideline  implementation  through  site  visits  as  well  as  e-mail  and 
phone  communication.  The  practice  changes  were  carried  out  by  the 
MTFs,  as  the  health-care  delivery  organizations.  The  MTFs  were 
offered  the  flexibility  to  define  strategies  and  clinical  process  changes 
within  the  context  of  their  respective  missions,  populations,  and 
administrative  and  clinical  assets.  Because  these  characteristics  dif¬ 
fered  across  facilities,  we  expected  to  observe  differences  among  the 
MTFs’  implementation  strategies  and  the  pace  at  which  they  intro¬ 
duced  practice  changes. 

This  chapter  reports  the  findings  of  the  implementation  process 
evaluation  in  terms  of  the  infrastructure  established  for  the  diabetes 
guideline  demonstration  and  the  strategies  and  actions  undertaken  by 
the  MTFs  to  implement  best  practices  for  management  of  diabetes 
patients.  We  first  describe  the  MEDCOM  support  structure  and 
activities,  followed  by  a  description  of  the  MTF  environment  and 
support  structure  for  guideline  implementation  at  the  participating 
MTFs.  Then  we  describe  the  strategies  and  actions  the  MTF  teams 
identified  in  their  implementation  action  plans.  Finally,  we  summa- 
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rize  the  lessons  learned  from  the  experiences  of  the  MTFs  participat¬ 
ing  in  this  demonstration. 

RAND’s  process  implementation  evaluation  was  conducted  at 
the  two  AMEDD  demonstration  MTFs.  A  summary  of  the  process 
evaluation  conducted  for  two  of  the  three  TRICARE  Senior  Prime 
demonstration  sites  is  also  included  at  the  end  of  the  chapter.' 


MEDCOM  Support 

The  corporate  responsibility  for  operating  the  AMEDD  program  for 
evidence-hased  practice  guidelines  was  assigned  to  the  MEDCOM 
Quality  Management  Directorate.  Initially,  the  staff  for  this  new  ini¬ 
tiative  consisted  of  a  full-time  program  director  and  a  secretary.  By 
the  time  the  diabetes  guideline  demonstration  began,  the  MEDCOM 
program  staff  had  been  expanded  with  the  addition  of  three  full-time 
guideline  representatives  who  supported  MTFs  in  implementing  the 
three  demonstrations  as  well  as  other  practice  guidelines  being 
implemented  across  the  AMEDD  system. 

The  diabetes  practice  guideline  demonstration  had  the  advan¬ 
tage  of  building  on  the  lessons  learned  from  the  low  back  pain  and 
asthma  guideline  demonstrations.  These  lessons  were  shared  with  the 
participating  MTFs  during  the  kickoff  conference  and  throughout 
their  work  during  the  demonstration.  The  MEDCOM  staff  contin¬ 
ued  to  be  committed  and  highly  motivated,  and  they  worked  collabo- 
ratively  with  the  MTF  teams  in  these  development  efforts. 

MEDCOM  supported  the  MTFs  in  implementing  the  diabetes 
practice  guideline  by  organizing  an  offsite  kickoff  conference  to 
introduce  the  implementation  teams  to  the  guideline  and  help  them 
develop  implementation  action  plans,  providing  the  MTFs  with  a 
toolkit  of  items  to  support  guideline  implementation,  providing 
information  systems  support  to  the  demonstration  sites,  and  facili¬ 
tating  communications  between  MTFs  and  MEDCOM  and  provid- 


*  We  did  not  have  access  to  the  notes  from  the  site  visit  to  the  third  facility. 
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ing  technical  support  to  the  MTFs.  We  describe  our  findings 
regarding  each  of  these  components. 

The  Kickoff  Conference 

The  implementation  teams  gathered  for  two  days  on  December  2-3, 
1999,  in  Tacoma,  Washington,  to  prepare  for  implementation  of  the 
guideline.  Upon  arrival  at  the  conference,  participants  were  given  a 
notebook  containing  information  on  the  guideline,  toolkit  items,  and 
instructions  for  preparing  an  implementation  action  plan.  The  con¬ 
ference  began  with  a  half-day  plenary  session  at  which  the  diabetes 
guideline  was  introduced  and  instructions  for  action  plan  develop¬ 
ment  were  provided.  For  the  remainder  of  the  conference,  the  teams 
met  separately  and  prepared  their  strategies  and  action  plans  for 
implementing  the  guideline  at  their  facilities.  Each  MTF  team  had 
designated  a  facilitator  who  guided  the  team  through  a  four-step 
planning  process  developed  by  RAND,  using  a  set  of  worksheets  for 
preparation  of  the  action  plan  document.  The  MTF  teams  briefed  the 
representative  of  regional  command  on  their  action  plans  at  the  end 
of  the  conference. 

One  of  the  lessons  from  past  experience  was  the  importance  of 
having  all  materials  completed  and  ready  for  the  participating  MTFs 
by  the  time  the  kickoff  conference  began.  Delays  in  previous  demon¬ 
strations  in  providing  toolkit  materials  and  key  metrics — and  in  one 
case  the  guideline  itself — made  it  difficult  for  the  MTFs  to  begin 
working  with  the  guideline  as  quickly  as  they  had  hoped.  For  the 
diabetes  kickoff  conference,  MEDCOM  had  completed  the  final 
practice  guideline,  list  of  key  guideline  elements,  key  metrics  for 
monitoring,  and  the  guideline  toolkits.  These  materials  were  distrib¬ 
uted  to  the  participating  MTEs  before  and  during  the  demonstration 
kickoff  conference. 

The  Diabetes  Toolkit 

In  preparation  for  the  diabetes  practice  guideline  demonstration, 
MEDCOM  and  CHPPM  convened  a  toolkit  workshop  on  Septem¬ 
ber  9,  1999.  At  this  workshop,  tools  were  developed  to  assist  Army 
MTEs  in  working  with  the  diabetes  practice  guideline.  Eour  work 
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groups  operated  during  the  workshop,  each  with  responsibility  for 
one  group  of  tools:  provider  support  tools,  patient  self-management 
support  tools,  and  system  support  tools  for  electronic  health  systems 
as  well  as  nonelectronic  record  systems.  Using  the  products  generated 
hy  the  workshop  teams,  MEDCOM  and  CHPPM  then  prepared  the 
core  set  of  items  to  he  included  in  the  initial  diabetes  practice  guide¬ 
line  toolkit.  The  tools  included  in  the  kit  are  shown  in  Table  4.1. 
They  were  tested  by  the  demonstration  MTFs. 

The  tools  identified  at  the  workshop  for  operational  support  and 
information  systems  were  not  “physical”  items  as  such,  but  entities 
such  as  patient  self-monitoring  system  (s),  a  model  disease  manage¬ 
ment  program,  and  methods  for  educating  clinic  staff  on  the  guide¬ 
line  and  related  practices.  Work  on  these  tools  continued  within 
MEDCOM  and  CHPPM.  Information  systems  tools  also  were  pur¬ 
sued,  including  an  automated  SF600  form  (the  Chronological  Record 
of  Medical  Care,  the  federal  government’s  standard  medical  form), 
several  options  for  a  diabetes  registry,  and  methods  for  identification 
of  at-risk  patients  from  automated  databases. 

Feedback  on  the  Toolkit.  During  the  first  postimplementation 
site  visits,  feedback  was  obtained  on  the  toolkit  items.  In  response  to 
the  feedback,  MEDCOM  and  CHPPM  revised  the  tools.  By  the  sec¬ 
ond  site  visit,  the  MTFs  had  received  most  of  the  revised  toolkit 
items  as  well  as  some  newly  developed  tools  that  had  been  requested 
in  the  first  site  visit.  The  feedback  was  particularly  helpful  because  it 
represented  the  contrasting  views  of  a  large,  specialty  facility  and  a 
small,  remote  community  hospital. 


Table  4.1 

Contents  of  the  Diabetes  Toolkit 


Provider  Support 

Patient  Self-Management  Support 

Encounter  documentation  (Form  705-R) 
Diabetes  flow  sheet  (Form  706-R) 

Guideline  key  elements  reminder  card 
Pocket-size  reminder  card 

Provider  education  briefing  slides 

Action  plan  workbook 

Patient  education  booklet 

Patient  education  video 
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Encounter  Documentation  Form.  Standardized  documentation 
methods  are  intended  to  ensure  that  all  the  key  information  required 
for  management  of  diabetes  care  is  efficiently  entered  into  the 
patients’  medical  charts.  To  this  end,  documentation  form  705-R  and 
diabetes  flow  sheet  706-R  were  developed  for  the  diabetes  toolkit. 

The  documentation  form  included  three  sections:  one  to  be 
completed  by  the  patient,  one  to  be  completed  by  the  clinic  staff,  and 
one  to  be  completed  by  the  physician.  At  our  first  site  visits,  both 
MTFs  reported  that  they  were  not  using  this  form  because  they  had 
identified  several  problems  with  it.  Problems  identified  included  lan¬ 
guage  in  the  patient  section  too  complex  for  patients  to  understand, 
lack  of  space  for  laboratory  results  and  medications  prescribed,  and 
lack  of  space  for  write-ins.  One  MTF  chose  to  use  the  form  705-R 
with  modifications,  and  some  providers  in  the  MTF  used  the  flow 
sheet  706-R.  The  other  MTF  chose  not  to  use  either  form  because 
the  staff  had  concerns  about  facing  too  many  forms  if  they  used  a 
separate  form  for  each  practice  guideline  being  implemented.  Instead, 
they  decided  to  continue  to  use  the  SF-600  form  to  document  care, 
with  specific  information  for  diabetes  patients  printed  on  it.  One 
MTF  had  developed  a  modified  form  to  replace  the  MEDCOM 
form. 

Both  MTFs  also  identified  the  difficulty  of  working  with  this 
type  of  documentation  form  for  patients  with  multiple  diagnoses 
because  the  form  is  designed  to  address  only  one  condition.  This  issue 
has  surfaced  in  all  three  of  the  practice  guideline  demonstrations.  It 
appeared  to  be  especially  difficult  with  diabetes  because  providers 
reported  that  most  diabetic  patients  have  multiple  diagnoses,  and  the 
form  does  not  have  enough  space  to  write  about  all  of  them. 

Diabetes  Flow  Sheet.  In  general,  positive  feedback  was  received 
on  the  flow  sheet,  including  comments  that  it  provided  a  good  mem¬ 
ory  log  of  care  over  time.  Some  concern  arose  about  the  time  required 
to  complete  the  sheet.  In  addition,  staff  members  were  unclear  about 
where  to  file  the  sheets  in  the  charts,  and  as  a  result,  the  sheets  were 
not  easily  found  and  retrieved.  Some  providers  readily  began  to  use 
the  sheets  while  others  chose  not  to  use  them. 
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Provider  Education  Briefing  Slides.  Few  comments  were  given 
on  the  provider  briefing,  which  was  an  adaptation  of  the  kickoff  con¬ 
ference  practice  guideline  briefing.  This  tool  apparently  had  little  use. 

Reminder  Cards  on  Guideline  Key  Elements  and  Coding.  These 
cards  were  accepted  as  containing  useful  information  for  providers. 
They  were  seen  as  particularly  useful  for  residents,  who  move  fre¬ 
quently.  Few  comments  were  offered  for  revisions. 

Patient  Education  Booklet.  Both  MTFs  offered  negative  feed¬ 
back  on  the  booklet.  They  reported  that  it  was  too  bulky,  too  busy, 
and  written  at  too  high  a  reading  level.  Further,  the  content  was  not 
detailed  enough  to  give  patients  needed  information.  At  one  MTF, 
patients  reportedly  did  not  like  the  tear  sheet  in  the  booklet.  Sug¬ 
gested  revisions  included  bringing  the  text  to  the  fifth-grade  reading 
level  and  creating  separate  handouts  for  each  topic. 

Patient  Education  Videos.  The  videos  were  viewed  as  acceptable 
by  the  MTFs,  although  one  MTF  did  not  use  the  adult  video  because 
they  considered  the  content  too  basic.  A  clinic  that  was  part  of  the 
other  MTF  did  not  use  the  video  because  it  lacked  the  equipment 
needed  to  show  it. 

Additions  to  the  Toolkit.  It  was  suggested  that  an  education 
program  be  developed  for  clinic  staff  that  would  cover  various  pro¬ 
cesses  to  support  effective  diabetes  care.  Items  that  needed  to  be  cov¬ 
ered  included  foot  exams,  use  of  equipment,  taking  blood  pressure, 
patient  education,  chart  review  methods,  and  ICD-9  coding. 

One  of  the  MTFs  had  developed  several  additional  patient- 
support  tools  that  could  be  shared  with  other  MTFs.  These  items 
included  posters  on  foot  care,  use  of  monofilaments  to  test  for 
peripheral  neuropathy,  and  diabetes  complications. 

Information  Exchange 

Two  mechanisms  were  established  to  help  the  MTFs  share  their 
implementation  experiences  and  learn  from  each  other  during  the 
demonstration — a  listserv  operated  through  the  AMEDD  e-mail  sys¬ 
tem  and  ongoing  interactions  with  and  feedback  from  MEDCOM 
during  the  demonstration  (which  continued  after  the  demonstration 
was  complete  as  a  routine  part  of  MEDCOM  support).  Again  draw- 
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ing  on  lessons  from  previous  demonstrations,  the  demonstration  par¬ 
ticipants  on  the  listserv  were  a  small  number  of  people  from  each 
demonstration  MTF  who  served  as  liaisons  between  MEDCOM  and 
the  local  MTF  implementation  teams.  In  addition,  the  MEDCOM 
guideline  office  staff,  national  diabetes  guideline  champions,  RAND 
team  members,  and  other  interested  parties  were  included. 

MEDCOM  conducted  periodic  teleconferences  or  videoconfer¬ 
ences  to  communicate  with  the  sites  during  the  demonstration. 
MEDCOM  staff  also  participated  in  the  two  rounds  of  site  visits  for 
the  RAND  evaluation,  during  which  they  were  able  to  address  ques¬ 
tions  from  the  sites  and  more  generally  assist  them  in  their  implemen¬ 
tation  activities.  In  addition  to  these  organized  encounters, 
MEDCOM  staff  were  in  regular  contact  with  staff  at  the  demonstra¬ 
tion  MTFs,  and  one  staff  member  had  designated  responsibility  to 
support  implementation  of  this  guideline. 


Structure  and  Support  at  the  MTFs 

The  MTF  Environment 

The  two  demonstration  MTFs  differed  substantially  in  their  sizes, 
clinical  capabilities,  previous  quality  improvement  activities,  and  use 
of  clinical  practice  guidelines.  Both  sites  had  the  basic  clinical  capa¬ 
bilities  for  the  treatment  of  diabetes,  including  primary  care  clinics, 
patient  education  resources,  and  at  least  some  relevant  specialty  care. 
The  extent  of  total  staffing  resources  available  to  undertake  this  initia¬ 
tive,  however,  was  a  big  factor  in  their  ability  to  implement  new 
processes  and  systems.  The  unique  features  of  the  two  MTFs  influ¬ 
enced  the  strategies  and  actions  chosen  by  their  teams  for  imple¬ 
menting  the  diabetes  practice  guideline. 

As  a  small,  remote  facility,  Bassett  ACH  was  understaffed  with 
respect  to  the  ratio  of  support  staff  to  providers,  and  it  was  located  at 
a  distance  from  specialized  medical  and  surgical  resources.  Virtually 
all  care  for  diabetic  patients  was  provided  in  the  internal  medicine 
clinic,  where  the  guideline  implementation  activities  were  under¬ 
taken.  An  emphasis  was  placed  on  establishing  stringent  standards  to 
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control  patients’  diabetes  because  of  the  geographic  barriers  to 
obtaining  specialized  care  for  diabetic  complications. 

Madigan  AMC  is  a  large  facility  that  serves  as  one  of  the  spe¬ 
cialized  regional  resources  for  the  more  remote  facilities.  The  adult 
primary  care  clinic  and  family  practice  clinic  saw  most  of  the  diabetic 
patients,  and  both  clinics  were  involved  in  the  demonstration. 
Reflecting  its  scope  of  resources,  Madigan  took  a  comprehensive 
approach  to  intervene  at  various  stages  of  diabetes  management,  with 
the  goal  of  improving  performance  on  the  DQIP  indicators  for  gly- 
cemic  control  and  tests  for  complications.  Before  this  demonstration, 
Madigan  had  been  an  active  member  of  a  statewide  diabetes  initiative. 

Support  for  the  Demonstration 

To  prepare  for  implementation  of  the  diabetes  guideline,  command¬ 
ers  of  the  demonstration  MTFs  were  requested  to  appoint  a  multi¬ 
disciplinary  implementation  team  of  eight  to  ten  individuals  who 
represented  the  mix  of  clinical  and  support  staff  involved  in  deliver¬ 
ing  care  for  patients  with  diabetes.  This  team  size  has  been  shown  to 
be  optimal  for  effective  team  operation.  The  implementation  team 
was  given  the  responsibility  to  develop  an  action  plan  for  strength¬ 
ening  diabetes  care  and  to  facilitate  its  implementation. 

The  commanders  also  were  requested  to  designate  a  guideline 
champion  and  a  facilitator  to  lead  the  implementation  activities.  The 
champion  was  the  leader  of  the  implementation  activities  and  the 
MTF  team.  The  facilitator  was  to  guide  the  implementation  team  in 
developing  an  implementation  action  plan  and  then  was  to  provide 
support  to  the  champion  and  team  in  coordinating  and  managing  the 
implementation  process. 

Command  Support  and  Accountability.  Commanders  at  the 
demonstration  MTFs  agreed  to  participate  in  the  diabetes  guideline 
demonstration.  The  support  of  the  MTF  commanders  continued 
over  the  life  of  the  demonstration,  despite  an  extensive  turnover  of 
command  staff  at  one  of  the  facilities.  In  addition,  staff  at  the  lead 
agent  office  of  TRICARE  Region  1 1  were  highly  supportive  of  the 
demonstration,  including  participation  in  the  kickoff  conference  and 
both  sets  of  site  visits  during  the  process  evaluation.  Their  goal  was  to 
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learn  from  this  process  so  it  could  be  implemented  more  broadly 
across  facilities  in  the  region. 

After  the  MTF  commanders  designated  guideline  champions, 
facilitators,  and  implementation  teams,  they  authorized  the  teams’ 
participation  in  the  two-day  offsite  conference  that  initiated  the  dem¬ 
onstration.  However,  none  of  the  leaders  or  members  of  the  teams 
were  formally  given  dedicated  time  to  devote  to  carrying  out  the 
guideline  action  plan.  Rather,  it  was  assigned  as  a  duty  in  addition  to 
other  tasks,  which  one  team  referred  to  as  “an  invitation  to  failure.” 

The  implementation  teams  at  both  MTFs  reported  regularly  to 
their  commands.  At  one  MTF,  the  champion  reported  directly  to  the 
MTF  leadership.  At  the  other,  the  activities  were  reported  as  part  of 
the  clinical  standards  reports  to  the  executive  board  of  directors,  con¬ 
sisting  of  the  MTF  clinical  leadership.  Such  an  emphasis  on  regular 
reporting  establishes  accountability  for  progress  in  implementing 
improved  diabetes  clinical  practices.  At  the  second  round  of  site  visits, 
both  MTF  commanders  made  clear  statements  in  support  of 
improved  diabetes  practices,  and  they  responded  to  issues  identified 
during  the  site  visit  with  directions  to  staff  to  address  the  problems 
and  report  results  to  them. 

The  Champions.  The  champion  at  one  MTF  was  an  internist, 
and  the  champion  at  the  other  MTF  was  an  endocrinologist  (Madi- 
gan  actually  employed  two  champions,  one  for  each  of  the  clinics). 
Both  made  strong  commitments  to  this  role  and  invested  substantial 
time  in  leading  the  work  on  implementing  their  diabetes  action  plans. 
Both  champions  had  to  fit  their  demonstration  responsibilities  into 
already  heavy  schedules.  They  saw  the  need  to  have  one  person  dedi¬ 
cated  to  such  an  effort  with  responsibility  to  seeing  that  the  defined 
actions  are  taken.  One  of  them  reported  that  a  champion  needs  to 
work  effectively  with  providers,  understand  the  subject  matter,  and 
know  the  process  of  care  being  used. 

The  Facilitators.  Both  demonstration  MTFs  had  facilitators  who 
had  been  actively  involved  in  the  implementation  process  since  before 
the  kickoff  conference.  One  facilitator  was  a  nurse  case  manager  and 
the  other  was  pathway  coordinator.  These  individuals  took  lead  roles 
in  coordinating  the  activities  and  tracking  measures  of  performance. 
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The  Implementation  Teams.  Both  MTFs  established  teams  that 
included  the  appropriate  clinical  and  support  staff,  including  primary 
care  providers  and  nursing,  plus  other  types  of  staff.  When  available, 
pharmacy,  health  education,  and  podiatry  also  were  represented  on 
the  teams.  One  MTF  started  with  a  20-member  team  that  met  on  an 
as-needed  basis,  communicating  between  meetings  by  e-mail  and 
informal  conversations.  They  found  this  group  was  too  large  to  coor¬ 
dinate  effectively  and  meet  on  a  regular  basis,  and  they  allowed  attri¬ 
tion  to  reduce  the  team  size  and  composition.  The  other  MTF  had  a 
small  team  of  four  or  five  persons  that  worked  together  informally 
without  scheduling  many  official  meetings. 


Implementation  Activities  and  Progress 

The  implementation  of  the  diabetes  guideline  began  in  early  April 

2000. 

Implementation  Strategies 

At  the  kickoff  conference,  the  teams  were  encouraged  to  approach 
implementation  by  undertaking  actions  on  a  small  scale  first,  through 
which  they  could  gain  experience  and  correct  problems  identified 
before  launching  a  major  change  in  practices  across  the  organization. 
Both  MTFs  took  this  approach,  working  with  one  or  two  clinics 
within  their  facilities  to  improve  practices  for  care  of  diabetes 
patients.  However,  their  approaches  to  introducing  practice  changes 
differed  greatly,  reflecting  the  substantial  differences  in  the  two 
MTFs. 

One  MTF  defined  a  comprehensive  action  plan  with  four  main 
components  designed  to  enhance  effective  management  of  diabetes 
patients:  establish  and  maintain  a  diabetes  registry  to  track  key  DQIP 
measures,  empower  the  ancillary  staff  through  education  to  effect 
change  in  patient  care,  employ  marketing  strategies  to  set  realistic 
patient  and  provider  expectations  regarding  diabetes  care,  and 
streamline  diabetic  assessment  and  management  through  use  of  algo- 
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rithms  and  standardized  documentation  of  care.  Implementation 
began  with  one  or  two  health-care  teams  in  each  of  two  clinics. 

The  other  MTF  planned  to  optimize  patient  management  and 
reduce  variability  in  provider  practices  for  diabetes  care,  with  the  goal 
of  standardizing  care  across  its  facility  and  the  VA  facility  with  which 
it  works.  To  achieve  this  goal,  patients  were  to  be  intimately  involved 
in  their  own  care.  Implementation  was  focused  in  the  internal  medi¬ 
cine  clinic,  which  sees  80  percent  of  the  diabetes  patients. 

The  Implementation  Process  and  Activities 

To  carry  out  their  respective  strategies,  the  sites  introduced  the 
guidelines  algorithm  and  supporting  toolkit  items  to  providers  and 
staff,  sought  to  make  changes  to  administrative  procedures,  provided 
patient  education  and  strategies  for  self-management,  and  monitored 
selected  indicators.  We  summarize  the  experiences  of  the  two  sites  in 
each  of  these  implementation  steps  and  discuss  the  various 
approaches  and  activities  they  undertook. 

Guideline  Introduction  and  Training.  Both  of  the  sites  began 
implementing  the  diabetes  guideline  with  education  of  providers  and 
clinic  staff  on  the  guideline,  the  DQIP  measures,  and  the  accompa¬ 
nying  toolkit.  Both  also  noted  that  they  recognized  the  need  for 
ongoing  education  and  for  integrating  guideline  orientation  into  edu¬ 
cation  for  newcomers. 

At  one  MTF,  an  orientation  session  was  held  with  command 
staff  and  the  chiefs  of  the  family  practice  and  adult  primary  care  clin¬ 
ics.  A  separate  in-service  training  session  was  held  with  the  primary 
care  providers  on  the  two  clinic  teams  selected  to  implement  the 
guideline.  At  these  sessions,  participants  were  presented  with  data 
from  initial  chart  reviews,  and  many  were  surprised  that  they  were 
not  performing  as  well  on  the  measures  as  they  had  believed.  Several 
areas  were  identified  where  additional  staff  training  was  needed, 
including  use  of  equipment  (e.g.,  insulin  pumps),  taking  blood  pres¬ 
sure,  chart  review,  ICD-9  coding,  and  patient  education.  Concerns 
were  raised  about  the  ability  of  clinic  staff  to  perform  the  cumulative 
functions  required  by  all  the  practice  guidelines. 
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At  the  other  MTF,  a  one-hour  continuing  medical  education 
session  was  held  for  providers,  including  nurses,  in  January  2000  that 
was  attended  hy  15  individuals.  In  this  session,  the  diabetes  algorithm 
was  reviewed  in  detail,  and  participants  identified  items  they  thought 
should  he  changed.  Clinic  staff  were  introduced  to  the  guideline  in  a 
separate  session,  where  implications  for  tasks  they  should  perform 
were  discussed.  Subsequent  education  focused  on  specific  issues, 
including  coding  of  diabetes  visits  and  management  of  newly  arrived 
patients. 

Changes  in  Practices  Used  to  Identify  Diabetic  Patients.  A  vari¬ 
ety  of  techniques  were  tested  by  the  demonstration  MTFs  to  improve 
the  procedures  used  to  manage  care  for  diabetic  patients.  An  early 
issue  to  be  resolved  was  how  to  identify  diabetic  patients  and  flag 
them  for  the  providers.  As  one  MTF  team  stated,  it  is  not  possible  to 
provide  adequate  diabetes  care  in  a  20-minute  appointment  if  the 
staff  does  not  know  in  advance  that  the  patient  has  diabetes.  At  one 
MTF,  charts  were  tagged  with  a  color-coded  sticker.  In  addition,  each 
evening,  nurses  reviewed  the  appointment  list  for  the  following  day  to 
identify  diabetes  patients  and  attach  the  proper  forms  to  the  chart.  At 
the  other  MTF,  diabetic  patients  were  assigned  to  a  specific  provider, 
and  case  management  was  provided  for  complex  or  difficult  cases. 
Both  MTFs  instituted  procedures  to  prepare  patients  and  prompt 
providers  to  perform  foot  exams. 

Patient  Education.  The  two  demonstration  MTFs  took  substan¬ 
tially  different  approaches  to  patient  education,  reflecting  differences 
in  the  size  of  patient  populations  served.  One  MTF  had  decentralized 
diabetes  patient  education  that  included  informal  education  in  the 
clinics  provided  by  clinic  staff  along  with  formal  education  provided 
separately  by  a  diabetic  educator,  a  nutritionist,  a  clinical  pharmacist, 
and  a  wellness  center.  The  diabetes  educator  was  to  see  all  new  diabe¬ 
tes  patients,  initially  for  a  one-on-one  session  and  then  in  groups  once 
a  week  for  three  weeks.  The  nutritionist  strove  to  see  all  patients,  but 
some  patients  missed  or  failed  to  make  appointments.  Difficult  cases 
were  referred  to  a  clinical  pharmacist  for  case  management.  The  well¬ 
ness  center  provided  education  in  coping  techniques  for  patients 
having  trouble  adjusting  to  living  with  diabetes.  The  patient  educa- 
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tion  booklet  in  the  toolkit  was  used  and  was  reportedly  well  received 
by  the  patients. 

The  other  MTF  developed  a  more  centralized  patient  education 
approach,  in  which  clinics  refer  patients  for  either  one-on-one  diabe¬ 
tes  education  or  nutrition  education.  The  education  emphasized  get¬ 
ting  the  patient  involved  in  self-monitoring.  The  staff  believed  the 
best  results  were  obtained  by  an  initial  45-minute  session  followed  by 
two  or  three  shorter  education  visits.  The  MTF  also  had  a  wellness 
clinic  that  provided  some  education  for  diabetic  patients,  but  this 
education  was  not  completely  coordinated  with  the  in-depth  educa¬ 
tion  provided  by  the  diabetes  educator.  This  MTF  used  its  own 
printed  materials  for  patient  education  rather  than  the  booklet  in  the 
toolkit. 

Monitoring  and  Feedback.  Both  of  the  MTFs  identified  the 
need  for  a  diabetes  registry  that  would  maintain  clinical  records  for 
each  diabetic  patient  served  by  the  MTF.  This  resource  would  sup¬ 
port  providers  in  monitoring  glycemic  control  and  ensuring  that 
screening  for  complications  and  organ  system  involvement  was  per¬ 
formed  on  schedule.  One  MTF  began  work  on  a  registry,  with  lim¬ 
ited  progress  during  the  time  of  the  demonstration.  Delays  in 
development  appeared  to  stem  from  competing  demands  for  staff 
time  as  well  as  some  confusion  in  the  specifications  for  such  a  system. 
The  other  MTF  had  planned  to  develop  a  registry  of  diabetic  patients 
in  a  spreadsheet  format  but  could  not  do  so  because  of  resource  con¬ 
straints  and  delays  caused  by  data  system  problems.  As  of  the  end  of 
the  demonstration,  its  staff  were  still  working  on  establishing  the  data 
capability  to  develop  this  registry  and  to  monitor  progress  in  manag¬ 
ing  diabetes  care. 

At  the  start  of  the  demonstrations,  AMEDD  did  not  have  the 
policy  framework  or  information  system  infrastructure  in  place  to 
establish  an  Army-wide  diabetes  registry,  but  the  topic  of  registries 
was  addressed  at  the  1 999  toolkit  workshop.  As  a  result,  MEDCOM 
initiated  efforts  to  support  local  registry  development  that  would 
adapt  either  the  San  Diego  Naval  Hospital  system  or  the  Triplet 
AMC  system  for  use  at  Madigan  AMC.  Both  systems  drew  similarly 
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upon  CHCS  and  ADS  data  to  compile  information  that  providers 
received  as  they  saw  diabetic  patients. 

With  respect  to  monitoring,  one  MTF  extensively  monitored  its 
performance  on  six  of  the  DQIP  indicators,  including  measurement 
of  indicators  at  two  points  in  time  before  start  of  guideline  imple¬ 
mentation  and  one  measurement  later  in  the  demonstration.  Thus, 
the  MTF  could  track  trends  in  improvements  on  the  DQIP  indica¬ 
tors  as  a  result  of  its  service  interventions.  Data  for  measurement  were 
obtained  monthly  from  laboratory  data  and  medical  chart  extraction. 
Thirty  charts  were  extracted  each  month,  with  data  entered  into  a 
personal  digital  assistant  (PDA).  Information  was  reported  routinely 
to  the  executive  board  of  directors  by  the  quality  management  func¬ 
tion.  Audits  currently  are  reported  at  the  clinical  team  level,  but  the 
staff  believes  they  will  not  get  provider  buy-in  until  data  are  reported 
at  the  provider  level. 

The  other  MTF  had  not  yet  generated  systematic  monitoring 
data  as  of  the  end  of  the  demonstration,  although  a  10  percent  draw 
of  medical  records  for  diabetes  patients  had  been  performed  to  check 
documentation  of  care.  Through  this  activity,  they  verified  that  80 
percent  of  the  diabetes  patients  were  in  the  internal  medicine  clinic 
and  that  charts  for  these  patients  documented  care  well.  The 
remaining  patients  were  in  the  family  practice  clinic.  Their  charts 
were  generally  good,  although  some  were  poorly  documented.  Staff 
were  preparing  a  second  set  of  chart  audits  at  the  time  of  our  second 
site  visit.  This  MTF  was  also  arranging  for  data  download  stations  in 
the  clinics  and  pharmacy,  with  an  ad  hoc  program  being  written  by 
the  pharmacist  to  extract  the  needed  data  elements. 


Highlights  of  Implementation  by  the  Tricare  Senior  Prime 
Demonstration  Sites 

This  section  summarizes  the  implementation  process  evaluation  for 
the  two  TRICARE  Senior  Prime  demonstration  sites  in  AMEDD 
Region  6. 
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Actions  Taken  to  Implement  Practice  Improvements 

As  occurred  with  the  two  MTFs  in  the  AMEDD  demonstration,  the 
two  Region  6  MTFs  pursued  different  strategies  to  achieve  similar 
results. 

One  MTF  undertook  inpatient  screening,  development  and  use 
of  a  self-report  screen  tool,  use  of  the  automated  SF-600  form  with 
preprinted  laboratory  history  and  hlood  pressure,  and  use  of  a  stan¬ 
dard  foot  exam  document  and  flow  sheet.  In  addition,  the  MTF  did 
periodic  provider  education,  marketing  of  diabetes  self-care  in  the 
community,  and  patient  education.  This  MTF  identified  issues  that 
needed  further  work,  including  staffing  constraints  that  limited 
patient  access  to  diabetes  management  or  education  programs,  foot 
clinics,  and  ophthalmology.  Additional  issues  that  were  documented 
were  missing  data  for  some  outcome  measures  and  lack  of  standardi¬ 
zation  of  tools  to  document  diabetes  care  in  the  medical  charts. 

The  other  MTF  established  a  comprehensive  management  pro¬ 
cess  to  improve  continuity  and  efficacy  of  care,  and  it  revised  its 
patient  education  program  to  support  that  process  as  well  as  to  meet 
national  standards.  Issues  that  this  MTF  identified  for  further  work 
included  variability  in  treatment  plans  and  the  need  to  improve  drug 
treatment,  patient  education,  and  ancillary  treatment.  Attention  was 
also  given  to  improving  coding  accuracy  in  the  diabetes  database, 
achieving  more  complete  chart  and  laboratory  review,  contacting 
patients  with  missing  or  high  values  for  diagnostic  tests,  providing 
feedback  to  providers  on  patients  not  meeting  process  goals,  and 
implementing  an  initiative  to  improve  foot  care. 

Challenges  Stemming  from  External  Factors 

The  diabetes  guideline  implementation  teams  for  the  Region  6  MTFs 
identified  a  number  of  external  factors  that  substantially  hampered 
their  ability  to  achieve  their  intended  practice  improvements: 

•  Implementation  teams  were  not  given  resources  to  support 

guideline  implementation. 
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•  No  Standard  database  was  available  for  diabetes  patients,  and 
computer  capabilities  for  screening  subgroups  of  patients  were 
limited. 

•  Coding  was  problematic  because  DoD  does  not  enforce  system- 
wide  standard  codes. 

•  Education  classes  by  noncredentialed  providers  are  not  given 
manpower  credits. 

•  Patients  had  limited  access  to  education  classes  if  they  were  not 
enrolled  in  TRICARE. 


Lessons  Learned:  Conclusions  and  Recommendations 
Regarding  Implementation 

MEDCOM  Support 

•  Development  and  implementation  of  clinical  practice  guidelines 
is  a  major  undertaking  that  requires  a  significant  commitment  of 
staff  resources.  The  decision  by  MEDCOM  to  provide  proactive 
corporate  support  to  the  MTEs  for  their  guideline  implementa¬ 
tion  activities  was  welcomed  by  the  MTEs  participating  in  this 
demonstration.  As  more  guidelines  are  introduced,  the  MTEs 
may  be  expected  to  turn  yet  more  frequently  to  the  MEDCOM 
staff  for  support  and  technical  assistance. 

•  The  question  of  whether  and  how  to  use  standard  documenta¬ 
tion  forms,  which  had  arisen  in  the  previous  demonstrations  for 
the  low  back  pain  and  asthma  practice  guidelines,  was  again  an 
issue  for  the  diabetes  guideline.  Neither  MTE  used  the  standard 
forms  as  provided  but  recognized  the  need  for  standardizing 
documentation  of  care.  A  solution  is  needed  that  allows  effective 
documentation  with  provision  for  management  of  patients  with 
multiple  conditions. 

•  Effective  monitoring  of  performance  in  diabetes  care,  as  well  as 
care  for  other  conditions,  requires  use  of  consistent  coding  of 
diagnoses  and  procedures  in  the  outpatient  encounter  records. 
Although  MEDCOM  has  established  standard  diabetes  codes, 
use  of  these  codes  has  been  less  than  complete,  at  least  in  part 
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because  DoD  does  not  enforce  consistent  coding  practices. 
Improvements  in  coding  pracdces  will  require  repeated  educa¬ 
tion  for  providers  and  clinic  staff,  along  with  feedback  on  per¬ 
formance  from  ADS  data. 

•  The  demonstration  MTFs  duplicated  the  experiences  of  MTFs 
in  previous  demonstrations  with  respect  to  difficulties  in 
extracting  and  working  with  the  local  data  needed  to  monitor 
performance  on  the  DQIP  indicators.  Given  the  current  limita¬ 
tions  of  the  military  health  system’s  automated  data  systems, 
MEDCOM  should  continue  to  provide  the  MTFs  with  techni¬ 
cal  support  to  help  them  establish  the  capabilities  to  monitor 
progress  on  the  DQIP  indicators  and  report  feedback  to  provid¬ 
ers. 

•  A  natural  extension  of  establishing  effective  data  capability  for 
diabetes  care  is  the  development  of  a  diabetes  registry.  Ideally,  a 
centralized  registry  should  be  developed  that  covers  all  diabetes 
patients  in  the  system  so  complete  information  on  diabetes 
patients  can  be  accessed  by  MTFs,  wherever  the  patients  may  be 
assigned  or  otherwise  located.  Lacking  that,  MEDCOM  should 
provide  needed  support  to  the  MTEs,  including  design  specifica¬ 
tions  and  programming  modules,  so  they  can  establish  local 
registries  for  patients  receiving  care  at  their  facilities. 

Support  at  the  MTF 

•  The  command  teams  of  the  participating  MTEs  provided  visible 
and  vigorous  support  for  practice  improvements  in  diabetes  care, 
making  it  clear  that  the  leadership  has  placed  a  priority  on 
achieving  the  best  practices  delineated  in  the  practice  guideline. 
This  support  gave  the  champions  and  implementation  teams  the 
authority  and  credibility  to  carry  out  their  action  plans.  The 
teams  also  were  asked  to  report  progress  on  practice  changes  and 
performance  on  key  indicators  to  the  MTE  leadership,  thus 
establishing  accountability  for  performance.  However,  these 
actions  were  not  accompanied  by  dedicated  financial  resources 
to  support  the  champions  and  teams  in  carrying  out  this  work. 
Participants  at  both  MTEs  reported  that  some  practice 
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improvement  actions  were  delayed  or  not  done  at  all  because  of 
these  resource  limitations. 

•  The  two  demonstration  MTFs  made  very  different  choices  for 
the  guideline  champion.  The  experiences  of  these  champions,  as 
well  as  those  of  many  others  chosen  to  lead  implementation 
efforts  for  the  low  back  pain  and  asthma  guidelines,  highlight 
that  the  champion’s  specialty  is  less  important  than  the  individ¬ 
ual’s  commitment  to  the  task  and  credibility  with  his  or  her 
peers.  Thus,  we  reiterate  here  our  findings  from  previous  dem¬ 
onstrations  that  the  champion  should  be  a  physician  who  is 
motivated  to  lead  the  process  of  changing  practices  according  to 
the  practice  guideline,  be  a  respected  opinion  leader  among  the 
providers,  and  have  military  rank  commensurate  to  those  of  his 
or  her  peers  at  the  MTF.  The  champion  also  should  be  known 
to  have  the  authority  to  make  needed  changes  to  procedures  and 
clinical  practices. 

Differences  in  the  two  MTF  implementation  teams  mirror  dif¬ 
ferences  across  MTF  teams  found  in  the  earlier  demonstrations.  The 
champion  and  facilitator  are  clearly  the  key  players  in  carrying  out  the 
guideline  action  plans,  but  the  composition  and  role  of  the  rest  of  the 
team  should  reflect  the  unique  service  mix  and  needs  of  each  MTF. 
Similarly,  regular  meetings  of  the  implementation  team  may  be  useful 
for  some  MTFs  but  less  desirable  for  others.  Regardless  of  meeting 
format,  strategic  involvement  by  team  members  helps  build  owner¬ 
ship  in  the  implementation  process  and  support  for  new  practices.  An 
agreed-on  mechanism  should  be  established  for  communications 
among  team  members  for  strategic  thinking,  troubleshooting  issues, 
and  assessment  of  progress. 


CHAPTER  FIVE 

Effects  of  Guideline  Implementation 


In  this  chapter,  we  examine  the  extent  to  which  introduction  of  the 
diabetes  practice  guideline  at  the  demonstration  MTFs  changed  clini¬ 
cal  practices  at  those  facilities,  as  measured  hy  selected  indicators.  We 
first  examined  trends  reported  by  the  demonstration  MTFs  in  the 
DQIP  performance  indicators  they  monitored.  Using  administrative 
data  from  the  demonstration  and  control  sites,  we  then  analyzed 
trends  in  service  utilization  for  the  five  identified  indicators  of  diabe¬ 
tes  care. 


Performance  Changes  Reported  by  MTFs 

As  the  two  AMEDD  demonstration  sites  and  two  TRICARE  Senior 
Prime  demonstration  sites  introduced  new  guideline-based  clinical 
practices,  they  all  monitored  the  effects  of  changes  in  their  perfor¬ 
mance  using  the  DQIP  indicators  described  in  Chapter  Two.  Because 
one  of  the  AMEDD  sites  and  both  of  the  TRICARE  Senior  Prime 
sites  had  been  tracking  their  progress  on  the  DQIP  measures  before 
the  demonstrations  began,  they  had  historical  trend  information  to 
serve  as  a  baseline. 

During  the  process  evaluation  site  visits,  the  demonstration 
MTEs  provided  information  on  which  indicators  they  were  moni¬ 
toring  (or  in  some  cases  preparing  to  monitor)  and  how  they  were 
performing  on  those  indicators.  As  shown  in  Table  5.1,  all  four 


65 


66  Implementation  of  the  Diabetes  Practice  Guideline  in  AMEDD 


Table  5.1 

DQIP  and  Other  Diabetes  Indicators  Monitored  by  the  Demonstration  MTFs 


Diabetes  Indicator 

Demo  1 

Demo  2 

Demo  3 

Demo  4 

DQIP — Glycemic  control 

Number  of  HbA,c  Hb,C-level  tests 
per  diabetic  patient 

Trend 

Planned 

Trend 

Trend 

Average  levels  of  HbA,c  for  dia¬ 
betic  patients 

Percentage  of  patients  with  HbA,c 
greater  than  9.5  percent 

Trend 

Trend 

Planned 

Trend 

Not 

reported 

DQIP — Prevent  complications 
Percentage  of  patients  assessed  for 
nephropathy 

Trend 

Planned 

Trend 

Trend 

Percentage  of  patients  with  a  lipid 
profile,  LDL  test  annually 

Trend 

Planned 

Trend 

Trend 

Percentage  of  patients  receiving  a 
dilated  eye  exam  at  least  annu¬ 
ally 

Trend 

Planned 

Baseline 

Trend 

Percentage  of  primary  care  visits  at 
which  patients  receive  a  foot 
exam 

Trend 

Planned 

Planned 

Trend 

Non-DQIP  indicators 

Percentage  of  patients  with  con¬ 
trolled  blood  pressure 

Trend 

Planned 

Planned 

Baseline 

Percentage  of  patients  with  docu¬ 
mented  diabetes  education 

Not 

reported 

Planned 

Baseline 

Baseline 

MTFs  were  collecting  the  DQIP  measures  or  were  planning  to  do  so. 
Three  of  the  MTFs  had  tracked  the  measures  over  time  and  had  trend 
data  for  at  least  two  points  in  time;  one  of  them  had  three  or  more 
years  of  data.  The  fourth  MTF  had  planned  to  monitor  the  measures 
through  a  patient  registry  it  was  developing,  hut  a  variety  of  imple¬ 
mentation  difficulties  slowed  progress  in  establishing  its  monitoring 
capability. 

In  general,  the  MTFs  with  trend  data  reported  that  their  per¬ 
formance  had  improved  on  the  indicators  they  tracked  from  the 
beginning  of  guideline  implementation. 

The  sources  of  data  for  self-reported  the  DQIP  indicators  were 
either  the  MTFs’  clinical  data  systems  (CHCS)  or  medical  charts. 
Data  on  completion  of  HbAjC,  urinary  microalbumin  and  albumin, 
and  lipid  profiles  were  available  in  the  MTFs’  CHCS  databases.  Data 
on  completion  of  foot  exams  had  to  be  obtained  from  the  medical 
charts.  The  eye  exam  indicator  data  could  be  obtained  from  medical 
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charts  or  from  encounter  data  for  ophthalmology  and  optometry 
clinics. 

MTF  data  collection  status  reports  revealed  problems  with  data 
quality  and  comparability.  The  descriptions  of  what  each  indicator 
actually  measured  were  poorly  documented  (e.g.,  percentage  of 
patients  getting  a  lipids  panel  versus  percentage  with  LDL  below  the 
specified  level).  Little  documentation  was  provided  for  how  each 
measure  was  calculated,  including  definitions  of  the  denominators 
and  numerators  for  the  calculations. 

Given  the  current  status  of  the  DoD  health  system  data  systems 
(see  Chapter  Four  for  discussion),  MEDCOM  faces  substantial  chal¬ 
lenges  to  its  ability  to  develop  a  system  for  monitoring  the  DQIP 
indicators  directly  at  the  system  level.  MEDCOM  does  not  have 
direct  access  to  the  local  MTE  data,  including  the  MTE  CHCS  data 
and  medical  charts.  However,  a  central  repository  for  pharmacy  data 
has  been  established  in  the  past  few  years,  and  one  for  laboratory  data 
is  under  construction.  When  complete,  these  databases  will  be  essen¬ 
tial  resources  for  tracking  many  quality  performance  indicators.' 

These  system-level  data  limitations  prevented  RAND  from 
relying  on  the  DQIP  measures  for  the  analysis  of  outcomes,  with  the 
exception  of  the  eye  exam  indicator,  for  which  ophthalmology  and 
optometry  clinic-encounter  data  were  available.  Thus,  as  described  in 
Chapter  Two,  it  was  necessary  to  identify  a  set  of  appropriate  indica¬ 
tors  of  guideline  implementation  for  which  administrative  data  were 
available  for  analysis.  The  remainder  of  this  chapter  reports  the  results 
of  these  analyses. 


Analysis  of  Effects  on  Service  Delivery 

Eor  the  analysis  of  trends  in  service  delivery,  we  compared  the  care 
practices  of  the  five  demonstration  sites  before  they  started  working 


'  While  not  a  component  of  this  guideline,  a  similar  capability  is  needed  for  radiology  data. 
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with  the  diabetes  practice  guideline  to  their  postimplementation  prac¬ 
tices  as  well  as  to  the  practices  of  the  five  control  sites. 

For  each  metric,  we  present  a  graph  of  trends  in  the  average  val¬ 
ues  for  the  demonstration  sites  and  control  sites.  For  the  measures  of 
primary  care  visits,  ER  visits,  and  hospitalizations,  values  are  reported 
for  each  quarter  in  the  two  years  of  the  study.  For  the  measures  of  use 
of  oral  hypoglycemic  agents  and  annual  eye  examinations,  we  use 
annual  values  reported  for  each  of  the  two  study  years.  Complete  data 
tables  are  included  in  Appendix  B.  We  also  tested  the  significance  of 
observed  trends  for  each  metric,  using  multivariate  regression  models. 
The  full  results  of  the  multivariate  modeling  also  are  presented  in 
Appendix  B. 

Use  of  Primary  Care  Services 

We  hypothesized  that  use  of  the  guideline  would  lead  to  an  increase 
in  primary  care  visits  in  the  first  part  of  the  demonstration,  as  the 
clinics  introduced  new  practices  to  actively  manage  care  for  diabetic 
patients.  We  also  hypothesized  that  such  an  increase  would  be  fol¬ 
lowed  by  a  decline  to  a  lower  steady  state  during  the  latter  part  of  the 
demonstration  period.  To  test  these  hypotheses,  we  examined  trends 
in  the  number  of  primary  care  visits  for  all  diabetic  patients  and  also 
separately  for  patients  who  used  insulin  and  those  who  did  not. 

The  baseline  primary  care  visit  rates  for  both  insulin-dependent 
and  non-insulin  dependent  diabetic  patients  were  higher  for  the 
demonstration  MTFs  than  for  the  control  MTFs  (Figure  5.1).  In  the 
last  two  quarters  of  the  demonstration  period,  primary  care  visit  rates 
decreased  for  the  demonstration  sites  while  those  for  the  control  sites 
did  not.  These  differences  were  statistically  significant  (see  Table  B.l 
in  Appendix  B) . 

No  short-term  increase  in  primary  care  visits  was  seen  early  in 
the  demonstration.  Such  an  increase  would  have  suggested  that  the 
demonstration  sites  were  bringing  in  diabetes  patients  more  frequent¬ 
ly  for  care  management  in  response  to  introduction  of  the  practice 
guideline.  Given  that  the  demonstration  sites’  primary  care  visit  rates 
were  higher  than  those  for  the  control  MTFs,  the  demonstration 
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Figure  5.1 

Trends  in  Primary  Care  Visits  for  All  Diabetic  Patients,  by  Demonstration  and 
Control  Sites 


RAND  MG277-5.1 


MTFs  may  already  have  had  appropriate  visit  rates.  However,  the 
data  do  not  show  whether  the  content  of  those  visits  changed. 

Several  reasons  could  exist  for  the  later  decline  in  primary  care 
use.  For  example,  growing  numbers  of  patients  might  have  under¬ 
taken  self-care  as  a  result  of  strengthened  patient  education  (a  desir¬ 
able  reason),  or  the  results  may  reflect  a  decline  in  monitoring  of  the 
patients  by  the  providers  (an  undesirable  reason).  These  results  are  a 
good  example  of  the  need  to  use  clinical  and  operational  information 
to  help  interpret  findings  from  administrative  data. 

The  same  trends  in  primary  care  visit  rates  were  seen  for  the 
insulin-dependent  and  the  non-insulin  dependent  patients  (Figure 
5.2).  However,  the  noninsulin  patients  have  consistently  lower 
primary  care  use  rates  than  those  using  insulin.  This  result  likely 
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Figure  5.2 

Trends  in  Primary  Care  Visits  for  Diabetic  Patients,  by  Insulin  User  or 
Nonuser  and  Demonstration  and  Control  Sites 


RAND  MG277-5.2 


reflects  the  more  severe  nature  of  insulin-dependent  diabetes  and  the 
greater  likelihood  of  the  presence  of  complications  or  comorbidities, 
which  require  greater  medical  supervision.  In  addition,  a  spike  in  vis¬ 
its  for  insulin-dependent  patients  happened  in  the  first  quarter  of  the 
demonstration  period,  which  did  not  occur  for  the  non-insulin 
dependent  patients. 

Use  of  Oral  Hypoglycemic  Agents  to  Control  Blood  Sugar 

The  diabetes  practice  guideline  places  an  emphasis  on  effective  gly- 
cemic  control,  which  has  been  shown  to  reduce  diabetic  complica¬ 
tions.  For  non-insulin  dependent  diabetic  patients,  we  hypothesized 
that  the  demonstration  MTFs  whose  baseline  practices  for  glycemic 
control  fell  short  of  those  specified  in  the  guideline  would  increase 
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the  use  of  oral  hypoglycemic  agents  (among  other  techniques)  to  con¬ 
trol  hlood  sugar. 

We  measured  the  use  of  oral  hypoglycemic  agents  as  the  per¬ 
centage  of  patients  who  were  prescribed  one  of  these  medications 
during  a  study  year  (see  Figure  5.3).  At  the  demonstration  sites,  the 
percentage  of  patients  who  filled  a  prescription  for  an  oral  agent  rose 
from  55.6  percent  in  the  baseline  year  to  58.4  percent  in  the  year 
following  guideline  implementation.  At  the  control  sites,  the  propor¬ 
tion  of  patients  who  filled  prescriptions  for  oral  agents  rose  from  48.1 
percent  during  the  baseline  year  to  57.8  percent  in  the  year  following 
implementation.  Regression  results  (reported  in  Tables  B.2  and  B.3 


Figure  5.3 

Percentage  of  Non-Insulin  Dependent  Diabetic  Patients  Who  Filled 
Prescriptions  for  Oral  Hypoglycemic  Agents,  by  Demonstration  and  Control 
Sites 


RAND  MG277-5.3 
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in  Appendix  B)  showed  that  the  increase  in  percentage  for  the  dem¬ 
onstration  sites  between  year  one  and  year  two  was  not  significantly 
different  from  the  increase  for  the  control  sites. 

Annual  Eye  Examinations 

The  diabetes  practice  guideline  recommends  that  all  patients  with 
moderate  to  severe  diabetes  have  eye  examinations  at  least  annually 
and  more  frequently  as  indicated.  The  purpose  of  the  eye  examina¬ 
tions  is  to  achieve  early  detection  and  treatment  of  any  effects  of 
diabetes  on  the  eye  to  avoid  or  delay  vision  loss.  We  examined  the 
proportion  of  diabetic  patients  enrolled  at  the  demonstration  MTFs 
who  had  at  least  one  annual  eye  examination.  As  described  in  Chap¬ 
ter  Two  and  Appendix  A,  we  defined  a  patient  as  having  an  annual 
eye  exam  if  the  patient  had  at  least  one  visit  to  an  optometry  or  oph¬ 
thalmology  clinic,  using  SADR  encounter  records  with  codes  for  a 
visit  to  one  of  these  clinics.  We  tabulated  the  total  number  of  patients 
with  at  least  one  visit  to  an  optometry  or  ophthalmology  clinic,  as 
well  as  the  subset  of  these  patients  for  which  diabetes  was  coded  as 
the  primary  diagnosis  for  at  least  one  of  their  visits,  which  we  defined 
as  having  a  diabetes-related  eye  examination. 

The  proportions  of  diabetic  patients  who  received  an  eye  exami¬ 
nation  during  baseline  were  10.6  percent  for  patients  at  the  demon¬ 
stration  MTFs  and  5.2  percent  for  control  patients  (Figure  5.4).  By 
study  year  two,  the  proportion  of  diabetic  patients  who  had  eye 
exams  had  increased  to  19.5  percent  at  the  demonstration  MTFs  but 
was  only  6.5  percent  at  the  control  MTFs. 

TVlthough  this  finding  suggests  that  introduction  of  the  practice 
guideline  contributed  to  increased  use  of  diabetes-related  eye  exami¬ 
nations  at  the  demonstration  sites,  our  analysis  suggests  that  the 
results  should  be  interpreted  quite  differently.  Figure  5.4  shows  the 
percentages  of  all  patients  (with  or  without  diabetes  as  a  primary 
diagnosis)  who  received  an  eye  examination,  as  well  as  the  percentage 
with  at  least  one  eye  examinations  for  whom  diabetes  is  coded  as  the 
primary  diagnosis  (reason  for  the  exam).  In  each  of  the  two  study 
years,  slightly  more  than  60  percent  of  all  patients  at  the  demonstra- 
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Figure  5.4 

Percentage  of  Diabetic  Patients  with  at  Least  One  Eye  Examination 
Annually,  for  Demonstration  and  Control  MTFs 


60  _ 


E 

03 

X 

0) 

Q) 

>. 

0) 


40  - 


0) 

O) 

03 

■H' 

c 

<u 


20  - 


■  Diabetes 
□  Other 


Year  1  Year  2 

Demonstration 


Year  1  Year  2 

Control 


RAND  MG277-5.4 


tion  MTFs  had  an  eye  examination,  and  the  only  increase  between 
the  two  study  years  was  in  the  percentage  of  patients  for  whom  the 
chart  indicated  the  examination  was  diahetes-related.  Therefore,  the 
apparent  increase  in  patients  with  diahetes-related  eye  examinations 
might  have  been  the  result  of  improved  coding  of  the  principal  diag¬ 
nosis  rather  than  an  increase  in  the  percentage  of  diabetic  patients 
who  had  eye  exams. 

An  alternative  explanation  is  also  possible  for  the  observed  shift 
in  eye  exams.  If  limited  staffing  and  physical  space  prevented  MTF 
eye  clinics  from  increasing  eye  exam  visits  in  response  to  growing 
demand,  an  increase  in  diahetes-related  eye  visits  could  have 
decreased  the  number  of  other  types  of  visits  offered. 

A  different  pattern  is  observed  for  the  control  MTFs.  Although 
the  total  percentages  of  diabetes  patients  with  eye  examinations 
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increased  from  approximately  35  percent  in  study  year  one  to  42  per¬ 
cent  in  study  year  two,  the  percentage  of  patients  with  diahetes- 
related  eye  examinations  showed  little  increase  between  the  two  years. 

Use  of  ER  Services 

We  measured  ER  use  as  the  number  of  ER  visits  per  100  diabetic 
patients  per  year.  Annualized  use  rates  were  calculated  for  each  quar¬ 
ter  of  the  two  study  years  to  allow  trends  to  be  examined.  As  shown 
in  figure  5.5,  patients  at  the  demonstration  sites  had  higher  ER-visit 
rates  than  those  at  the  control  sites.  Examination  of  trends  in  use 
showed  a  substantial  increase  from  the  baseline  to  the  postimplemen¬ 
tation  period  for  the  demonstration  sites,  while  use  rates  for  the  con¬ 
trol  sites  remained  stable  over  the  eight  quarters  of  the  study. 

Figure  5.5 

Trends  in  ER  Visits  per  100  Diabetic  Patients,  All  Demonstration  and 
Control  Sites 


RAND  MG277-5.5 
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Further  examination  of  rates  for  the  individual  MTFs  revealed 
that  the  apparent  increase  for  the  demonstration  sites  was  attributable 
to  increases  at  one  large  MTF.  We  could  not  determine  the  reason  for 
this  increase,  but  incomplete  data  on  ER  encounters  for  some  quar¬ 
ters  could  account  for  at  least  some  of  the  pattern  observed.  When 
data  for  this  MTF  were  removed  from  the  analysis,  the  rates  for  the 
remaining  four  sites  were  found  to  be  stable  over  time,  as  shown  in 
Figure  5.6.  The  regression  modeling  (Tables  B.5  and  B.6  in  Appen¬ 
dix  B)  found  no  statistically  significant  reduction  in  ER  visits  for 
these  four  demonstration  sites  when  compared  to  the  control  sites. 

Use  of  Hospital  Inpatient  Services 

Two  measures  were  used  to  assess  the  effects  of  introducing  the  prac¬ 
tice  guideline  on  hospitalization:  diabetes-related  hospitalizations 

Figure  5.6 

Trends  in  ER  Visits  per  100  Diabetic  Patients  (Demonstration  Site  3  Data 
Omitted),  Demonstration  and  Control  Sites 


RAND  MG277-5.6 
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(hospital  stays  with  diabetes  coded  as  the  principal  diagnosis)  and  all 
hospitalizations  (regardless  of  principal  diagnosis).  Annualized  hospi¬ 
talization  rates,  defined  as  the  number  of  admissions  per  1 00  diabetic 
patients,  were  calculated  for  each  quarter  of  the  two  study  years. 

As  shown  in  Figure  5.7,  diabetes-related  inpatient  stays  repre¬ 
sent  75  to  80  percent  of  total  inpatient  stays  for  the  patients  in  the 
study  sample.  The  percentages  of  diabetes-related  inpatient  stays  are 
similar  for  the  demonstration  and  control  MTFs  and  did  not  vary 
significantly  over  the  course  of  the  study. 

Diabetes  patients  at  the  demonstration  sites  had  higher  total 
hospitalization  rates  than  those  at  the  control  sites,  as  shown  in  Fig¬ 
ure  5.8.  Similar  baseline  patterns  were  found  for  diabetes-related 
hospitalization  rates  (see  Appendix  B).  Given  the  similar  trends  for 

Figure  5.7 

Trends  in  Diabetes-Related  or  Other  Hospital  Inpatient  Stays  per  100 
Diabetic  Patients,  Demonstration  and  Control  Sites 
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Figure  5.8 

Trends  in  Total  Hospital  Inpatient  Stays  per  100  Diabetic  Patients, 
Demonstration  and  Control  Sites 


RAND  MG277-5.8 


diabetes-related  and  total  hospitalization  rates  for  the  diabetic  patients 
in  our  sample,  we  used  the  total  hospitalization  rates  to  analyze  the 
effects  of  guideline  implementation. 

Trends  in  total  hospitalization  rates  were  stable  for  both  the 
demonstration  and  control  sites  (Figure  5.8).  The  regression  model¬ 
ing  (see  Table  B.7  in  Appendix  B)  found  no  statistically  significant 
change  in  hospitalization  rates  for  the  demonstration  sites  when  com¬ 
pared  to  the  control  sites. 


Estimated  Costs  of  Care  for  MTFs  in  the  Study 

We  began  the  cost  analysis  by  looking  at  the  total  costs  of  care  for  all 
diabetic  patients  served  by  the  ten  MTFs  included  in  the  study.  We 
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looked  at  costs  from  two  perspectives — the  mix  of  costs  experienced 
by  the  MTFs  for  both  enrollee  and  nonenrollee  diabetic  patients  and 
the  mix  of  MTF  costs  experienced  by  each  of  these  two  groups  of 
beneficiaries.^ 

A  substantial  share  of  the  MTF  costs  for  diabetic  patients  is  for 
nonenrollees,  as  shown  in  Figure  5.9.  MTF  primary  care  providers 
have  more  difficulty  managing  care  for  nonenrollees  than  for  enrol- 
lees  because  nonenrollees  would  use  the  services  of  an  MTF  at  which 
they  were  not  enrolled  only  on  an  episodic  basis.  Comparing  the 


Figure  5.9 

Composition  of  MTF  Total  Costs  for  Enrollee  and  Nonenrollee  Diabetic 
Patients  for  Demonstration  and  Control  Sites,  by  Study  Year 
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^  As  discussed  in  Chapter  Two,  some  uncertainty  exists  about  the  quality  of  MEPRS  cost 
data,  and  the  data  quality  varies  across  MTFs. 
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demonstration  and  control  sites,  the  costs  for  MTF  enrollees  were  a 
much  larger  share  of  total  costs  for  the  demonstration  MTFs  than  for 
the  control  sites.  This  difference  was  found  in  both  study  years. 

Figure  5.10  shows  estimated  average  MTF  costs  from  the 
patient  perspective,  including  total  costs  and  costs  for  outpatient  and 
inpatient  services  per  diabetic  patient.  Diabetic  patients  enrolled  at 
MTFs  had  lower  total  MTF  costs  per  patient,  on  average,  than  those 
using  the  MTFs  on  a  more  episodic  basis,  attributable  to  the  higher 
MTF  inpatient  costs  for  nonenrollees. 

Several  explanations  might  contribute  to  this  result.  The  enrol¬ 
lees  may  receive  more  regular  care,  which  would  prevent  conditions 
or  complications  that  lead  to  inpatient  stays  or  reduce  the  severity  of 
treatment  needed  during  an  inpatient  stay.  For  example,  enrollees 


Figure  5.10 

Estimated  Total  MTF  Cost  per  Patient  for  MTF  Enrollees  and  Other  Users, 
Demonstration  and  Control  Sites,  by  Study  Year  (in  1999  $) 
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admitted  as  inpatients  might  undergo  fewer  surgical  procedures, 
which  lead  to  more  costly  inpatient  stays.  One  might  also  speculate 
that  enrollees  obtain  relatively  more  inpatient  care  from  community 
providers  than  the  nonenrollees  do,  hut  this  explanation  is  not  likely 
because  enrollees  are  expected  to  obtain  care  at  the  MTF  where  they 
are  enrolled  unless  the  MTF  does  not  offer  the  services  needed. 

An  alternative  explanation  for  the  high  nonenrollee  inpatient 
costs  can  be  found  in  the  demographics  of  the  diabetic  patients  served 
by  MTFs.  These  patients  include  a  large  percentage  of  individuals 
over  65  years  of  age  who  are  eligible  for  Medicare  as  well  as  DoD 
health  benefits.  They  have  access  to  MTF  services  only  on  a  space- 
available  basis  because  Medicare  is  their  primary  payer.  Under  this 
DoD  policy,  MTF  inpatient  care  is  much  more  accessible  than  MTF 
outpatient  services  (because  the  MTFs  are  underfilled).  Further,  eld¬ 
erly  patients  might  be  more  likely  to  seek  MTF  inpatient  care  to 
avoid  the  large  out-of-pocket  costs  for  civilian  hospitals. 

An  exception  to  the  policy  of  restricting  access  to  MTF  care  for 
elderly  beneficiaries  was  the  TRICARE  Senior  Prime  demonstration, 
in  which  Medicare-eligible  beneficiaries  could  enroll  in  an  MTF- 
based  managed  care  plan  for  three  years.  This  demonstration  termi¬ 
nated  at  the  end  of  200 1 .  Three  of  the  demonstration  MTFs  included 
in  the  present  study  were  Senior  Prime  sites  at  the  time  of  the  study. 
Thus,  beneficiaries  enrolled  in  Senior  Prime  at  those  MTFs  had  pri¬ 
ority  for  MTF  care  equal  to  that  of  the  TRICARE  Prime  enrollees, 
and  these  beneficiaries  were  included  in  our  study  sample.  However, 
a  large  fraction  of  Medicare-eligible  beneficiaries  at  those  sites  chose 
not  to  enroll  in  Senior  Prime,  so  some  elderly  enrollees  still  used 
MTF  care  on  a  space-available  basis. 

Overall  Costs  of  MTF  Services 

We  also  compared  average  overall  costs  for  diabetes  care  for  study 
years  one  and  two.  Enrollees  (our  study  sample)  and  nonenrollees 
who  used  MTF  services  were  considered  separately. 

Costs  for  MTF  Enrollees.  The  aggregate  and  per-patient  MTF 
costs  for  diabetic  patients  enrolled  at  the  MTFs  are  presented  in 
Table  5.2.  The  costs  incurred  by  demonstration  MTFs  for  services  to 
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enrolled  diabetic  patients  increased  from  an  estimated  $14.9  million 
in  year  one  to  $16.7  million  in  year  two.  This  is  the  result  of  both  an 
increase  in  the  number  of  patients  (from  6,024  to  6,654  patients)  and 
a  slight  increase  in  per-patient  costs  (from  $2,477  to  $2,514). 

These  estimates  exceed  those  for  the  control  MTFs,  which 
incurred  an  estimated  $4.6  million  in  year  one  and  a  smaller  $4.2 
million  in  year  two.  The  much  smaller  costs  for  the  control  MTFs 
reflect  their  smaller  number  of  diabetes  patients,  less  than  half  the 
number  of  those  at  the  demonstration  MTFs,  as  well  as  substantially 
lower  costs  per  patient  ($1,659  and  $1,470  per  patient  in  years  one 
and  two,  respectively). 

For  the  demonstration  MTFs,  inpatient  services  represented 
approximately  two-thirds  of  the  total  estimated  costs  in  years  one  and 


Table  5.2 

Estimated  MTF  Costs  for  Diabetes  Patients  Enrolled  at  the 
Demonstration  and  Control  MTFs,  Study  Years  One  and  Two 


All  Sites 

Demonstration 

Sites 

Control  Sites 

Study  year  one 

Number  of  patients 

8,779 

6,024 

2,755 

Aggregate  costs 

Total 

$19,489,575 

$14,920,468 

$4,569,109 

Outpatient 

$5,194,683 

$3,963,963 

$1,230,720 

Inpatient 

$14,294,892 

$10,956,505 

$3,338,389 

Costs  per  patient 

Total 

2,220 

2,477 

1,659 

Outpatient 

592 

658 

447 

Inpatient 

1,628 

1,819 

1,212 

Study  year  two 

Number  of  patients 

9,495 

6,654 

2,841 

Aggregate  costs 

Total 

$20,905,006 

$16,727,238 

$4,177,768 

Outpatient 

$5,922,413 

$4,433,434 

$1,488,979 

Inpatient 

$14,982,593 

$12,293,804 

$2,688,789 

Costs  per  patient 

Total 

2,202 

2,514 

1,470 

Outpatient 

624 

666 

524 

Inpatient 

1,578 

1,848 

946 

NOTE:  All  costs  are  in  1999  dollars  that  were  estimated  by  adjusting  1998 
unit  cost  estimates  by  an  inflation  factor  of  1.4  percent. 
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two.  By  comparison,  for  the  control  sites,  inpatient  services  were 
about  three-quarters  of  total  costs  for  year  one,  but  in  year  two,  the 
costs  shifted  such  that  a  relatively  larger  share  was  attributable  to  out¬ 
patient  services. 

The  introduction  of  new  care  management  practices  for  control 
of  blood  sugar  and  prevention  of  complications  would  be  expected  to 
increase  outpatient  care  use  rates  and  related  costs  for  the  demonstra¬ 
tion  MTFs.  As  shown  in  Table  5.2,  the  average  outpatient  cost  per 
patient  for  diabetic  patients  enrolled  at  the  demonstration  MTFs 
increased  by  1.2  percent  from  baseline  to  year  two  (the  difference 
between  $666  and  $658).  At  the  same  time,  however,  the  average 
per-patient  outpatient  cost  for  enrollees  at  the  control  MTFs 
increased  by  17.2  percent.  The  cost  increase  for  the  demonstration 
MTFs  should  be  assessed  relative  to  the  increase  for  the  control 
MTFs,  which  represents  the  temporal  trend  for  costs.  Thus,  the 
minor  increase  in  costs  at  the  demonstration  MTFs  relative  to  that  of 
the  control  sites  is  unexpected. 

When  we  consider  these  outpatient  cost  increases  together  with 
our  earlier  findings  for  primary  care  visits,  the  increase  in  costs 
appears  to  reflect  increased  use  of  services  other  than  primary  care. 
Because  we  found  no  increase  in  primary  care  visits  for  either  the 
demonstration  or  control  MTFs,  it  appears  that  patients  in  both 
groups  of  MTFs  obtained  a  larger  volume  of  care  from  specialty  clin¬ 
ics  in  year  two  than  they  did  in  year  one. 

This  trend  should  be  monitored  to  assess  the  mix  of  services 
provided  and  how  they  relate  to  practice  guideline  recommendations. 
Increased  outpatient  costs  per  patient,  at  least  for  the  first  few  years 
after  introduction  of  the  guideline,  could  be  consistent  with  what  the 
guideline  defines  as  effective  practices. 

The  small  increase  in  estimated  inpatient  care  costs  for  the  dem¬ 
onstration  MTFs  (from  $1,819  to  $1,848  per  patient)  suggests  that 
use  of  the  diabetes  practice  guideline  did  not  yet  have  an  observable 
effect  on  inpatient  service  costs  for  the  MTFs  during  the  first  year 
following  guideline  introduction.  According  to  our  hypothesis,  such 
an  effect  would  have  decreased  the  per-patient  cost  as  more  effective 
care  management  began  to  reduce  avoidable  hospitalizations. 
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One  possible  explanation  for  the  absence  of  cost  reduction  could 
be  that  new  practices  need  to  be  in  use  for  longer  than  one  year 
before  they  achieve  observable  effects  on  reducing  diabetic  complica¬ 
tions  that  lead  to  hospitalization.  Another  factor  that  might  be  influ¬ 
encing  this  outcome  for  the  demonstration  sites  is  the  growth  in 
number  of  diabetes  patients  between  the  two  study  years.  The  addi¬ 
tional  patients  in  year  two  probably  represent  new  enrollees  who  had 
obtained  care  at  other  MTFs  in  the  previous  year,  so  their  health 
status  and  service  needs  would  reflect  the  care  management  they  had 
at  their  previous  MTF  more  than  the  care  being  provided  by  the 
demonstration  MTFs. 

Costs  for  Nonenrollees.  The  costs  of  care  for  nonenrollees  using 
the  demonstration  MTFs  are  presented  in  Table  5.3.  Total  costs  for 


Table  5.3 

Estimated  MTF  Costs  for  Nonenrollee  Diabetes  Patients  Using  Care 
at  the  Demonstration  and  Control  MTFs,  Study  Years  One  and  Two 


All  Sites 

Demonstration 

Sites 

Control  Sites 

Study  year  one 

Number  of  patients 

9,773 

3,768 

6,027 

Aggregate  costs 

Total 

$38,748,747 

$14,720,636 

$24,028,111 

Outpatient 

$5,374,949 

$1,886,865 

$3,488,084 

Inpatient 

$33,373,798 

$12,833,771 

$20,540,027 

Costs  per  patient 

Total 

3,965 

3,907 

3,987 

Outpatient 

550 

501 

579 

Inpatient 

3,415 

3,406 

3,408 

Study  year  two 

Number  of  patients 

9,646 

3,871 

5,810 

Aggregate  costs 

Total 

$38,030,234 

$13,526,996 

$24,503,237 

Outpatient 

$5,723,582 

$1,800,861 

$3,922,721 

Inpatient 

$32,306,652 

$11,726,135 

$20,580,516 

Costs  per  patient 

Total 

3,942 

3,494 

4,217 

Outpatient 

593 

465 

675 

Inpatient 

3,349 

3,029 

3,542 

NOTE:  All  costs  are  in  1999  dollars  that  were  estimated  by  adjusting  1998  unit 
cost  estimates  by  an  inflation  factor  of  1.4  percent. 
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the  demonstration  MTFs  were  an  estimated  $14.7  million  in  year 
one,  declining  to  $13.5  million  in  year  two.  At  the  same  time,  total 
costs  for  the  control  MTFs  increased  from  $24.0  million  to  $24.5 
million  for  the  two  years.  Inpatient  services  accounted  for  most  of  the 
total  costs  of  care  for  nonenrollees  for  both  the  demonstration  and 
control  MTFs. 

Outpatient  care  costs  for  nonenrollees  at  the  demonstration  sites 
decreased  from  baseline  to  year  two  by  7.2  percent  (the  difference 
between  $501  and  $465  per  patient),  while  these  costs  increased 
slightly  for  the  control  MTFs.  This  reduction  might  reflect  changes 
in  the  way  the  demonstration  MTFs  are  serving  their  enrollees  versus 
nonenrollees,  or  it  could  simply  be  a  fluctuation  in  care  needs  that 
would  not  be  sustained  in  following  years. 

The  inpatient  costs  of  care  for  nonenrollees  declined  at  the 
demonstration  MTFs  from  baseline  to  year  two  by  11.1  percent  (the 
difference  between  $3,406  and  $3,029  per  patient),  while  these  costs 
increased  slightly  for  the  control  MTFs  (Table  5.3).  This  reduction 
could  also  reflect  changes  in  the  care  the  demonstration  MTFs  are 
providing  their  enrollees  versus  nonenrollees,  or  it  could  be  a  normal 
annual  fluctuation  in  care  needs  and  related  costs. 

Variations  in  Costs  Across  Facilities 

We  found  substantial  variation  among  the  individual  MTFs  in  the 
costs  of  care  per  patient  during  the  baseline  year  as  well  as  the  year 
following  implementation.  The  total  and  per-patient  outpatient  and 
inpatient  costs  for  enrolled  diabetic  patients  are  shown  in  Tables  5.4 
and  5.5,  respectively. 

Most  of  the  ten  MTFs  in  the  study  had  average  annual  baseline 
outpatient  care  costs  of  less  than  $900  per  patient  (the  majority  being 
less  than  $600).  However,  costs  at  two  demonstration  MTFs  exceed¬ 
ed  $1,000  per  patient  (Table  5.4).  In  year  two,  the  costs  for  these  two 
MTFs  remained  high  (although  one  decreased  slightly) ,  and  the  costs 
for  two  other  MTFs  rose  to  greater  than  $1,000  per  patient.  At  the 
same  time,  per-patient  costs  declined  for  four  of  the  other  MTFs. 
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Table  5.4 

Estimated  Costs  of  Outpatient  Services  for  Diabetes  Patients  Enrolled 
at  the  Demonstration  and  Control  Sites,  Study  Years  One  and  Two 


Study  Year  One 

Study  Year  Two 

Site 

Number  of 

Patients  Total  Cost 

Cost  per 
Patient 

Number  of 

Patients  Total  Cost 

Cost  per 
Patient 

Demo  1 

2,178 

$542,566 

$249 

2,428 

$578,817 

$238 

Demo  2 

111 

$119,925 

$1,080 

128 

$123,774 

$967 

Demo  3 

2,149 

$2,206,692 

$1,027 

2,216 

$2,243,071 

$1,012 

Demo  4 

680 

$360,819 

$531 

825 

$348,036 

$422 

Demo  5 

906 

$733,961 

$810 

1,057 

$1,139,736 

$1,078 

Control  1 

1,339 

$567,743 

$424 

1,429 

$562,545 

$394 

Control  2 

159 

$25,030 

$157 

96 

$22,341 

$233 

Control  3 

351 

$305,904 

$872 

367 

$420,361 

$1,145 

Control  4 

489 

$157,725 

$323 

591 

$339,591 

$575 

Control  5 

417 

$174,318 

$418 

358 

$144,141 

$403 

NOTE:  All  costs  are  in  1999  dollars,  which  were  estimated  by  adjusting  1998 
unit  cost  estimates  by  an  inflation  factor  of  1.4  percent. 
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Estimated  Costs  of  Inpatient  Services  for  Diabetes  Patients  Enrolled 
at  the  Demonstration  and  Control  Sites,  Study  Years  One  and  Two 


Study  Year  One  Study  Year  Two 


Site 

Number  of 

Patients  Total  Cost 

Cost  per 
Patient 

Number  of 

Patients  Total  Cost 

Cost  per 
Patient 

Demo  1 

2,178 

$5,067,249 

$2,327 

2,428 

$5,607,460 

$2,309 

Demo  2 

111 

$53,219 

$479 

128 

$130,230 

$1,017 

Demo  3 

2,149 

$4,649,888 

$2,164 

2,216 

$4,612,302 

$2,081 

Demo  4 

680 

$563,871 

$829 

825 

$867,628 

$1,052 

Demo  5 

906 

$622,277 

$687 

1,057 

$1,076,184 

$1,018 

Control  1 

1,339 

$2,093,011 

$1,563 

1,429 

$1,510,587 

$1,057 

Control  2 

159 

$6,558 

$41 

96 

$8,731 

$91 

Control  3 

351 

$648,698 

$1,848 

367 

$743,063 

$2,025 

Control  4 

489 

$229,981 

$470 

591 

$244,126 

$413 

Control  5 

417 

$360,141 

$864 

358 

$182,283 

$509 

NOTE:  All  costs  are  in  1999  dollars,  which  were  estimated  by  adjusting  1998 
unit  cost  estimates  by  an  inflation  factor  of  1.4  percent. 


The  individual  MTFs  also  varied  in  the  levels  of  average  inpa¬ 
tient  care  costs  per  diabetic  patient  and  the  extent  to  which  those 
costs  changed  from  the  baseline  to  the  year  after  guideline  implemen¬ 
tation  (Table  5.5).  The  costs  for  the  two  demonstration  MTFs  with 
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the  largest  numbers  of  patients  declined  in  study  year  two,  while 
those  for  the  three  smaller  MTFs  increased.  Almost  the  reverse  pat¬ 
tern  was  found  among  the  control  sites. 


Summary 

Outpatient  visits:  The  number  of  outpatient  primary  care  visits  per 
100  patients  was  slighdy  higher  for  the  demonstrarion  sites  than  for 
the  control  sites  throughout  the  study  period  and  did  not  change  for 
either  group. 

Use  of  oral  hypoglycemic  agents:  The  proportion  of  diabetic 
patients  who  fdled  prescriptions  for  oral  hypoglycemic  agents 
increased  slightly  but  not  significantly  from  year  one  to  year  two  in 
both  demonstration  and  control  sites. 

Eye  exams:  At  the  start  of  the  study,  the  proportions  of  patients 
who  received  annual  diabetes-related  eye  exams  was  low  for  both  the 
demonstration  and  control  sites,  although  twice  as  many  demonstra¬ 
tion  patients  received  the  exams  as  control  patients.  At  year  two,  the 
proportion  of  demonstration  patients  receiving  diabetes-related  eye 
exams  doubled  (although  the  total  number  of  eye  exams  remained  the 
same)  whereas  the  proportion  of  control  patients  remained  the  same. 

Use  of  ER  services:  At  the  start  of  the  study,  demonstration 
patients  had  greater  use  of  ER  services  than  did  control  patients.  ER 
use  by  demonstration  patients  increased  throughout  the  study  period, 
whereas  use  rates  for  control  patients  did  not  change. 

Use  of  hospital  inpatient  services:  Diabetes-related  inpatient 
stays,  which  represented  the  majority  of  inpatient  service  use  for  the 
diabetic  population,  were  similar  between  the  demonstration  and 
control  sites  and  did  not  vary  significantly  over  the  course  of  the 
study. 

Costs  of  care  for  diabetic  patients:  As  a  proportion  of  total  costs 
of  diabetic  care  per  patient  and  per  MTE,  costs  of  care  for  nonenrol- 
lees  was  substantial  at  both  demonstration  and  control  hospitals. 
Nonenrollee  inpatient  costs  far  exceeded  enrollee  inpatient  costs. 
Many  of  the  nonenrollees  were  over-65  Medicare  recipients. 
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For  enrollees,  per-patient  costs  at  demonstration  hospitals 
exceeded  those  of  control  hospitals  for  both  inpatient  and  outpatient 
care  and  in  both  study  years.  From  year  one  to  year  two,  average  per- 
patient  costs  for  both  outpatient  and  inpatient  care  increased  slightly 
at  the  demonstration  sites,  while  at  the  control  sites,  outpatient  costs 
rose  slightly  and  inpatient  costs  fell. 

For  nonenrollees,  per-patient  costs  at  demonstration  hospitals 
were  comparable  to  or  slightly  less  than  those  of  control  hospitals  for 
both  study  years.  From  year  one  to  year  two,  average  per-patient  costs 
for  both  outpatient  and  inpatient  care  fell  slightly  at  demonstration 
sites,  while  at  the  control  sites,  outpatient  and  inpatient  costs  rose 
slightly. 

From  one  MTF  to  another,  per-patient  costs  for  both  outpatient 
and  inpatient  services  varied  widely.  It  should  be  noted  that  based  on 
our  analyses,  which  did  not  control  for  other  variables  that  might 
impact  costs  of  care  (such  as  level  of  service  use,  etc),  it  is  not  possible 
to  conclude  that  the  observed  changes  costs  noted  above  can  be 
attributed  to  the  implementation  of  the  diabetes  guideline. 


CHAPTER  SIX 


Syntheses  of  Findings  from  the  Demonstration 


As  the  last  of  three  demonstrations  designed  to  field-test  methods  for 
implementing  clinical  practice  guidelines,  the  diabetes  guideline 
demonstration  hoth  confirmed  findings  from  earlier  demonstrations 
and  highlighted  new  issues  regarding  management  practices  for  diabe¬ 
tes.  We  observed  the  success  of  the  participating  MTFs  in  establish¬ 
ing  viable  implementation  strategies  and  actions,  which  was  a  clear 
improvement  on  earlier  demonstrations,  while  also  observing  chal¬ 
lenges  and  barriers  shared  with  the  earlier  demonstrations.  The  MTFs 
participating  in  this  effort  were  able  to  draw  on  lessons  from  the  low 
back  pain  and  asthma  practice  guideline  demonstrations,  as  well  as 
their  own  experiences  in  implementing  other  practice  guidelines  that 
had  been  introduced  across  the  system  by  the  time  this  demonstration 
began. 

In  this  chapter,  we  will  first  discuss  the  implications  of  the 
findings  regarding  service  use  and  costs.  Second,  we  will  provide  some 
observations  regarding  the  factors  influencing  the  successes  and  limi¬ 
tations  of  the  diabetes  guideline  demonstration,  by  examining  how 
well  the  demonstration  performed  on  the  six  critical  success  factors 
presented  in  Chapter  One  of  this  report.  We  will  also  discuss  how  the 
implementation  activities  affected  the  DQIP  measures  and  other  rele¬ 
vant  service  delivery  measures  for  diabetes  patients.  Third,  we  will 
discuss  study  limitations.  Finally,  we  will  provide  some  recommenda¬ 
tions. 
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Implications  of  Findings  on  Service  Use  and  Cost 

The  combination  of  the  metrics  measured  by  the  demonstration 
MTFs  and  our  analysis  of  measures  in  this  study  provides  useful 
information  for  assessing  how  much  progress  was  made  during  the 
demonstration  in  implementing  the  diabetes  practice  guideline.  The 
DQIP  indicators  represent  the  most  important  set  of  measures  of  the 
performance  of  diabetes  care  practices.  However,  given  the  status  of 
the  military  health  data  systems,  most  of  these  indicators  are  best 
measured  locally  by  the  MTFs,  and  some  can  only  be  measured  there. 

The  indicators  of  service  delivery  activities  that  RAND  used  in 
this  study  complement  the  DQIP  measures  by  examining  some  of  the 
processes  of  care  that  can  be  measured  with  administrative  data. 
Observed  performance  on  these  indicators  can  provide  information 
about  the  underlying  processes  that  might  be  contributing  to  per¬ 
formance  on  DQIP  measures. 

As  we  test  for  effects  of  guideline  implementation,  we  are  guided 
by  the  sequence  of  effects  that  is  likely  to  occur.  The  first  changes  that 
should  be  observed  are  in  the  processes  and  procedures  used  to  deliver 
care,  which  is  where  the  MTFs  were  focusing  their  implementation 
strategies.  As  new  procedures  are  institutionalized,  which  often 
requires  time,  changes  should  be  observed  in  the  indicators  for  clini¬ 
cal  care  results,  such  as  reductions  in  avoidable  health  care  events. 

MTF  Progress  on  the  DQIP  Measures.  Four  of  the  five  MTFs 
included  in  our  effects  analysis  reported  they  were  working  with  the 
DQIP  measures.  Three  of  the  MTFs  had  tracked  the  measures  over 
time  and  reported  that  their  performance  had  improved  on  the  indi¬ 
cators  they  tracked  between  baseline  and  12  months  into  the  demon¬ 
stration.  Such  improvements  over  time  could  lead  to  a  reduction  in 
diabetes  complications  and  associated  avoidable  health-care  events 
(e.g.,  ER  visits  or  hospitalizations).  However,  documentation  of  the 
DQIP  data  and  comparability  across  MTFs  suggests  that  these  data 
will  be  of  limited  value  to  MEDCOM  for  systemwide  reporting  and 
MTE  comparisons. 

Effects  Found  in  the  RAND  Analysis.  The  RAND  analysis  found 
few  effects  associated  with  introduction  of  the  diabetes  practice  guide- 
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line.  Primary  care  visit  rates  for  the  demonstration  MTFs  held  steady 
in  the  first  two  quarters  of  the  demonstration  and  then  decreased  in 
the  last  two  quarters,  compared  to  the  control  MTFs.  Use  of  non¬ 
insulin  glycemic  control  medications  increased  from  baseline  during 
the  demonstration  period,  as  hypothesized,  hut  this  increase  did  not 
differ  significantly  from  that  of  the  control  MTFs. 

We  found  a  significant  increase  in  the  percentage  of  patients 
with  diahetes-related  annual  eye  examinations.  Further  analysis  sug¬ 
gested,  however,  that  this  increase  was  the  result  of  improved  coding 
for  the  diabetes  diagnosis  on  the  encounter  records,  rather  than  an 
actual  increase  in  annual  eye  examinations.  Even  so,  coding 
improvement  is  a  positive  result  because  accurate  coding  is  essential 
to  effective  performance  measurement.  For  ER  visit  rates  and  hos¬ 
pitalization  rates — indicators  that  represent  potentially  avoidable 
health-care  events — we  found  no  change  in  rates  during  the  demon¬ 
stration,  instead  of  the  declines  that  would  be  expected  as  a  result  of 
improved  care  management. 

Given  the  limitations  of  the  system-level  encounter  data,  the 
only  DQIP  measure  that  RAND  could  incorporate  into  its  evaluation 
was  the  annual  eye  exam  indicator  because  data  could  be  obtained 
from  ophthalmology  and  optometry  clinic  encounters.  The  other  five 
indicators  used  in  the  evaluation  were  measures  of  service  delivery  for 
diabetic  patients.  In  our  analysis  of  effects  on  service  delivery  indica¬ 
tors,  we  found  few  effects  of  the  practice  guideline  for  the  demonstra¬ 
tion  MTEs. 

DQIP  performance  data  provided  by  three  of  the  demonstration 
MTEs  suggested  improvements  in  many  of  the  DQIP  measures.  This 
improvement  contrasts  with  the  virtual  absence  of  improvement  in 
the  indicators  analyzed  by  RAND,  suggesting  that  RAND’s  measures 
were  unable  to  capture  process  changes  made  by  the  MTEs  that 
resulted  in  the  reported  improvements  on  the  DQIP  indicators.  The 
likely  reason  for  this  discrepancy  is  that,  with  the  exception  of  eye 
exam  performance  rates  (measured  as  optometry  or  ophthalmology 
clinic  visits) ,  we  counted  only  total  visits  and  were  unable  to  measure 
the  performance  of  any  types  of  tests  within  those  visits.  Another  pos¬ 
sibility  is  that  the  DQIP  performance  data  provided  by  the  MTEs 
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may  not  have  been  interpreted  accurately.  Some  inconsistencies  in 
definitions  and  reporting  methods  suggest  the  likelihood  of  this  pos¬ 
sibility. 

Acting  on  this  information  from  the  demonstration,  MED- 
COM  and  PASBA  initiated  Army-wide  monitoring  of  DQIP  metrics 
down  to  the  provider  level  with  aggregation  to  the  clinic,  MTF,  and 
MEDCOM  levels.  MTEs  could  gain  access  to  data  on  a  monthly 
basis.  MEDCOM  encountered  a  number  of  definitional  and  data 
quality  issues  with  metrics.  MEDCOM  now  has  made  the  transition 
to  DoD-supported  guideline  metric  monitoring. 

Variations  Across  MTFs.  When  we  looked  at  variations  across 
individual  MTFs,  we  found  large  differences  among  the  ten  MTFs, 
both  in  their  baseline  performance  levels  and  in  the  extent  to  which 
their  performance  changed  from  the  first  to  second  study  years.  These 
variations  highlight  both  major  differences  in  MTFs  (that  cannot  be 
accounted  for  without  additional  information)  and  the  fact  that  each 
MTF  implemented  different  practice  improvement  strategies  and  pri¬ 
orities,  reflecting  their  unique  operational  environments,  patient  mix, 
personnel  and  technological  resources,  and  physical  plants,  none  of 
which  could  be  captured  by  the  indicators  used. 

Data  System  Issues.  Both  feedback  from  the  MTFs  regarding 
data  system  problems  and  our  experience  working  with  the  adminis¬ 
trative  data  in  our  analyses  highlight  important  barriers  that 
MEDCOM  will  need  to  address  before  it  can  establish  an  efficient 
and  effective  monitoring  system.  These  issues  include  lack  of  integra¬ 
tion  of  individual  data  systems,  poor  documentation  of  systems  and 
procedures,  inability  to  extract  data  from  systems,  and  difficulties  in 
modifying  coding  or  automated  forms. 

Three  specific  data  issues  created  particular  challenges  for  dem¬ 
onstration  MTFs. 

Inconsistent  Coding  of  Diagnoses  and  Procedures.  Effective 
monitoring  of  performance  in  treating  diabetes  (and  for  other  condi¬ 
tions)  requires  consistent  coding  of  diagnoses  and  procedures  in  the 
outpatient  encounter  records.  MEDCOM  has  established  standard 
codes  for  diabetes,  but  these  codes  have  not  been  used  consistently 
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across  MTFs,  at  least  in  part  because  DoD  does  not  enforce  consis¬ 
tent  coding  practices. 

During  the  data  analysis,  several  coding  issues  were  identified 
that  affect  the  reliability  of  the  data.  First,  a  large  percentage  of 
patients  were  coded  inconsistently  for  diabetes  type  (i.e.,  they  were 
coded  as  Type  1  on  some  records  and  Type  2  on  others).  When  the 
patients  who  were  consistently  coded  as  Type  1  were  matched  to 
pharmaceutical  data,  only  a  small  percentage  of  these  patients  had 
records  for  insulin  prescriptions,  suggesting  that  the  pharmaceutical 
data  provided  for  our  study  were  incomplete.  Further,  the  percentage 
of  inconsistently  coded  patients  who  had  records  for  insulin  prescrip¬ 
tions  was  larger  than  expected,  suggesting  that  this  group  probably 
included  some  Type  1  diabetics. 

This  finding  raises  a  larger  question  about  the  accuracy  of  the 
data  used  to  test  trends  for  the  indicators  in  this  study,  as  well  as  con¬ 
cerns  regarding  the  acceptability  of  the  DoD  administrative  data  for 
ongoing  monitoring  of  diabetes  management  practices.  The  challenge 
for  MEDCOM  will  be  to  determine  how  to  handle  the  coding  of 
diabetes  types  in  administrative  data  to  best  provide  valid  and  com¬ 
plete  data  for  ongoing  monitoring  of  care  and,  ultimately,  for  a  diabe¬ 
tes  registry.  In  addition,  issues  related  to  data  completeness  were 
highlighted  by  the  clinical  laboratory  discrepancies. 

For  any  kind  of  centralized  monitoring  capability,  standardized 
coding  of  patient  status  or  procedures  is  required.  The  demonstration 
sites  have  made  progress  in  defining  standardized  codes  to  identify 
patients,  status  of  condition,  and  specific  procedures.  However,  these 
codes  have  not  been  used  consistently  by  all  sites,  with  the  result  that 
the  data  aggregated  at  the  system  level  for  these  variables  may  not  be 
consistent  and  should  be  used  with  caution. 

Difficulty  Extracting  Data  from  MTF  Data  Systems.  The  demon¬ 
stration  MTFs  faced  various  challenges  in  extracting  and  working 
with  the  data  needed  to  monitor  performance  on  the  DQIP  indica¬ 
tors,  ranging  from  undocumented  data  files  to  obtaining  space  on  the 
local  servers.  The  MTFs  are  likely  to  continue  to  need  technical  sup¬ 
port  from  MEDCOM  to  help  them  establish  the  needed  data  capa¬ 
bilities  locally.  Alternatively,  MEDCOM  could  establish  a  centralized 
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system  to  collect  the  data  directly  from  automated  data  systems,  per¬ 
form  analyses,  and  generate  trend  reports  to  the  MTFs. 

Absence  of  a  Diabetes  Registry.  DoD  has  no  diabetes  registry  in 
place,  and  none  of  the  demonstration  MTFs  had  a  centralized  registry 
for  diabetic  patients  in  their  systems.  Thus,  it  was  not  possible  to 
obtain  complete  information  on  diabetic  patients  or  to  track  them 
after  they  left  a  particular  MTF’s  care. 

Cost  Analysis.  Our  cost  analysis  provided  summary  information 
on  total  MTF  costs  for  services  to  diabetic  patients,  including  an 
assessment  of  the  shares  of  MTF  costs  attributable  to  patients 
enrolled  at  the  MTFs  and  to  those  who  used  MTF  services  but  were 
not  enrolled.  Two  key  findings  emerged.  First,  we  found  that  a  sub¬ 
stantial  share  of  MTF  costs  for  diabetic  patients  during  the  study 
years  were  incurred  for  patients  not  enrolled  at  the  MTFs,  especially 
for  use  of  inpatient  services.  This  finding  has  implications  for  how 
best  to  serve  these  nonenrollees,  considering  both  issues  of  care  man¬ 
agement  for  episodic  users  of  the  facilities  and  efficient  use  of  out¬ 
patient  and  inpatient  resources. 

The  second  key  finding  from  the  cost  analysis  was  the  small 
increase  in  per-patient  costs  from  year  one  to  year  two  for  enrolled 
patients  at  the  demonstration  MTFs,  while  costs  decreased  for  the 
control  MTFs.  This  result  suggests  that  early  changes  in  diabetes  care 
practices  by  the  demonstration  MTFs  may  have  increased  visit  rates 
and  costs  of  care  for  enrolled  patients  served  by  those  MTFs.  It  will 
be  important  to  track  inpatient  use  rates  and  costs  over  time  to  assess 
the  extent  to  which  early  cost  increases  may  reverse  themselves  and 
longer  term  effects  begin  to  be  observed  after  new  care  management 
methods  are  in  place  for  a  while.  As  cost  information  accumulates  for 
several  years,  it  also  will  be  possible  to  distinguish  trends  related  to 
practice  changes  from  normal  fluctuations  in  health-care  needs  from 
year  to  year. 

Finally,  we  note  that  some  criticism  was  heard  within  DoD  that 
the  MEPRS  data  overestimate  the  MTFs’  costs  of  doing  business. 
The  source  of  this  criticism  is  a  reported  lack  of  documentation  of 
vacation  time,  as  well  as  overestimation  of  the  available  hours  of  mili¬ 
tary  personnel  time  for  patient  care  activities  because  personnel  often 
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do  not  record  time  they  spend  on  military-related  activities.  While 
acknowledging  this  issue,  we  also  understand  that  MEPRS  offers  the 
best  available  data,  and  it  is  the  basis  for  all  other  cost  estimations  for 
the  demonstration. 


Guideline  Implementation:  Performance  on  Critical 
Success  Factors 

Research  on  the  practice  guideline  implementation  has  documented 
that  a  strong  commitment  to  the  implementation  process,  including 
use  of  multiple  interventions  targeted  at  identified  barriers,  is 
required  to  achieve  desired  changes  to  clinical  practices.  In  Chapter 
One,  we  outlined  six  critical  success  factors  that  have  been  found  to 
be  essential  for  success  in  making  lasting  changes  in  clinical  and 
administrative  processes.  We  discuss  here  the  extent  to  which  this 
demonstration  realized  these  success  factors,  and  we  assess  implica¬ 
tions  for  progress  in  implementing  practice  improvements. 

•  Command  leadership  commitment  at  the  MTF,  regional, 
and  corporate  levels.  This  demonstration  provides  a  positive 
example  of  how  leadership  commitment  can  support  the  ability 
to  achieve  practice  improvements.  In  this  case,  the  implementa¬ 
tion  teams  had  the  support  of  both  the  MTF  commands  and  the 
leadership  of  the  TRICARE  Region  1 1  Eead  Agent  office.  The 
lead  agent  staff  participated  in  the  demonstration  as  observers, 
with  the  intent  of  implementing  this  approach  in  other  MTFs  in 
the  region. 

•  Monitoring  of  progress.  The  performance  of  the  demonstra¬ 
tion  MTFs  in  the  area  of  monitoring  was  mixed.  One  MTF 
actively  measured  trends  in  performance  on  the  DQIP  measures, 
while  another  MTF  struggled  to  extract  the  needed  data  in  the 
face  of  inadequate  staffing  levels  and  technical  problems  with  its 
data  system.  Both  of  the  MTFs  performed  chart  reviews  to  assess 
current  documentation  of  care  and  to  extract  data  on  key  DQIP 
measures,  with  differing  success  in  establishing  a  routine  process 
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for  data  extraction.  Both  MTFs  also  strove  to  establish  a  local 
diabetes  registry,  with  limited  success. 

•  Guidance  and  support  to  the  MTFs  by  MEDCOM.  By  the 
time  the  diabetes  guideline  demonstration  began,  MEDCOM 
had  well-established  staffing  and  other  resources  and  was  pro¬ 
viding  policy  guidance  and  technical  support  to  help  MTFs 
implement  practice  improvements  for  diabetes  care.  Such  sup¬ 
port  also  can  encourage  movement  toward  consistency  in  prac¬ 
tices  across  the  Army  facilities.  We  believe  this  committed 
support  by  MEDCOM  has  been  a  strong  foundation  for  the 
practice  improvements  achieved  in  the  guideline  demonstra¬ 
tions,  as  MEDCOM  learned  from  each  field  test  and  applied 
those  lessons  to  subsequent  demonstrations. 

•  Guideline  champions  who  are  opinion  leaders.  The  partici¬ 
pating  MTFs  identified  well-respected  physicians  to  serve  as 
guideline  champions  for  the  diabetes  demonstration,  and  these 
physicians  showed  a  commitment  to  leading  the  implementa¬ 
tion  activities  for  their  facilities.  However,  this  demonstration 
(as  did  the  two  before  it)  showed  that  the  champions  could 
make  only  a  time-limited  commitment  to  the  initiative,  after 
which  they  tired  of  the  concentrated  effort  or  had  to  turn  their 
attention  to  other  priorities.  This  finding  highlights  the  impor¬ 
tance  of  integrating  new  practices  into  ongoing  procedures  as 
quickly  and  effectively  as  possible. 

•  Resource  support  for  champions.  Both  of  the  MTF  com¬ 
manders  authorized  the  champions  to  lead  the  implementation 
of  the  diabetes  guideline,  but  few  of  the  champions  received 
tangible  resource  support  for  their  activities  (other  than  atten¬ 
dance  at  the  kickoff  conference).  In  general,  they  had  to  perform 
the  implementation  work  in  addition  to  their  regular  workload. 
In  both  MTFs,  a  facilitator  designated  by  the  MTF  commander 
provided  some  staff  support  to  the  champion,  and  this  role  was 
an  integral  part  of  the  facilitator’s  regular  job. 

•  Institutionalization  of  new  practices.  The  participating  MTFs 
made  progress  in  achieving  practices  consistent  with  the  diabetes 
guideline,  focusing  on  areas  where  their  performance  on  DQIP 
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measures  was  the  weakest.  Sustainment  of  these  changes  is  likely 
because  they  addressed  clinically  important  issues,  and  both 
providers  and  clinic  staff  were  involved  in  implementing  the 
practice  changes. 


Study  Limitations 

A  number  of  factors  may  have  contributed  to  the  apparently  minor 
effect  of  the  demonstration.  Issues  pertaining  to  data  were  discussed 
above.  Other  factors  are  discussed  briefly  here. 

•  We  interacted  directly  with  only  two  of  the  five  demonstration 
MTFs  and  with  none  of  the  control  MTFs.  Therefore,  we  had 
little  operational  information  to  help  explain  observed  patterns 
or  trends  in  the  indicators  for  the  MTFs  in  the  study  sample. 
Furthermore,  we  included  in  the  analysis  three  TRICARE  Sen¬ 
ior  Prime  MTFs,  which  were  not  fully  participating  in  the 
evaluation  and  may  have  used  very  different  implementation 
methods. 

•  The  number  of  actual  demonstration  sites  was  small,  and  those 
two  sites  differed  greatly  with  respect  to  size,  location,  staffing, 
and  types  and  numbers  of  patients  served.  These  sites  may  not 
have  been  truly  comparable  to  the  control  sites. 

•  The  inability  to  track  individual  patients  from  baseline  to  year 
two,  coupled  with  the  difficulty  in  establishing  continuity  of 
care  when  personnel  move  frequently  or  when  nonenrolled 
retirees  obtain  only  some  of  their  care  at  a  particular  MTF  may 
have  decreased  the  likelihood  of  our  observing  real  changes. 

•  A  demonstration  is  an  artificial  situation.  By  definition,  it  is  the 
first  field  attempt  to  work  with  new  practices  and  lacks  the  full 
authority  of  a  program  officially  implemented  across  an  entire 
system. 

•  The  one-year  follow-up  time  allowed  before  postimplementation 
data  were  collected  is  too  short  to  expect  to  find  meaningful 
changes  in  performance  on  many  of  the  measures  of  interest  for 
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quality  improvement  programs.  Alternatively,  changes  in  care 
processes  may  have  been  occurring  (e.g.,  during  primary  care 
visits),  hut  the  indicators  we  used  could  not  capture  those 
changes. 

•  Finally,  it  is  possible  that  implementation  of  other  care  improve¬ 
ment  efforts  prior  to  the  start  of  the  demonstration,  of  which  we 
were  not  aware,  may  have  prevented  our  observing  any 
measurable  improvement. 


Recommendations 

Although  the  MTFs  participating  in  the  diabetes  practice  guideline 
demonstration  had  some  notable  successes  in  some  aspects  of 
improving  diabetes  treatment  practices,  resource  limitations  and 
organizational  barriers  curbed  the  overall  progress  made  in  the 
demonstration.  Of  particular  concern  was  the  inability  to  transfer 
gains  made  in  the  clinics  that  first  worked  with  the  guideline  to  other 
clinics  within  the  MTF.  Provided  here  are  some  additional  lessons 
learned  and  recommendations. 

Implementation 

•  The  MTFs  established  quite  different  implementation  strategies, 
which  reflected  each  MTF’s  unique  capabilities  and  circum¬ 
stances.  We  believe  this  flexibility  helps  ensure  that  each  MTF 
can  address  the  clinical  practices  most  in  need  of  improvement. 
However,  this  approach  puts  more  responsibility  on  each  MTF 
for  defining  its  own  direction,  and  it  also  may  slow  progress 
toward  the  AMEDD  goal  of  achieving  consistent  practices 
across  its  facilities. 

•  Provision  of  additional  resources  to  support  implementation 
activities  would  help  the  champions  and  teams  achieve  lasting 
improvements  in  practices. 

•  To  ensure  these  improvements  sustain  themselves,  regular  edu¬ 
cation  sessions  will  be  necessary  for  providers,  clinic  staff,  and 
newcomers  to  the  MTF.  Regular  feedback  to  providers  on  per- 
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formance  trends  for  the  DQIP  measures  will  also  help  to  rein¬ 
force  new  practices. 

•  Each  of  the  guideline  demonstrations  huilt  on  lessons  from  the 
ones  that  preceded  it,  and,  as  in  the  asthma  demonstration,  the 
diabetes  demonstration  continued  to  yield  useful  insights. 
MEDCOM  should  take  actions  to  strengthen  its  system  in 
response  to  the  lessons  identified  in  the  process  evaluation  for 
this  demonstration.  Indeed,  reports  from  MEDCOM  staff  con¬ 
firm  that  many  of  the  issues  identified  have  been  addressed  in 
subsequent  activities  as  several  practice  guidelines  have  been 
implemented  across  the  Army  health  system. 

Benchmarking  of  MTF  Performance 

•  The  assessment  of  baseline  service  use  for  the  demonstration  and 
control  sites  highlights  the  great  variability  in  practices  across  the 
MTEs.  MEDCOM  and  the  MTEs  should  use  this  baseline 
information  as  an  integral  part  of  the  regular  monitoring  for 
effective  diabetes  care,  to  identify  facilities  at  greatest  variance 
from  established  standards  and  identify  factors  contributing  to 
the  variance.  Interventions  should  be  undertaken  to  correct 
identified  performance  problems.  This  type  of  benchmarking 
system  not  only  provides  monitoring  information  to  the  system 
and  MTEs,  but  it  also  can  provide  an  empirical  foundation  that 
MEDCOM  can  used  to  guide  selection  of  performance  priori¬ 
ties  and  measures  for  future  years. 

Outcomes  Measurement 

•  MEDCOM  should  continue  to  document  variations  in  per¬ 
formance  on  key  indicators  across  MTEs  on  a  regular  basis  to 
identify  areas  where  improvements  in  quality  and  greater  consis¬ 
tency  are  needed.  With  this  information  in  hand,  MEDCOM 
can  determine  whether  to  give  the  MTEs  more  specific  direction 
with  regard  to  which  aspects  of  the  guideline  are  to  be  imple¬ 
mented  uniformly. 

•  It  is  important  to  institute  a  set  of  indicators  that  are  widely  in 
use  across  the  country,  including  instructions  on  how  to  calcu- 
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late  the  measures.  Monitoring  progress  toward  best  practices  was 
a  primary  focus  of  the  participating  MTFs.  Working  with  the 
DQIP  measures,  they  created  an  environment  of  data-driven 
implementation  and  accountability.  This  approach  helped  the 
MTFs  focus  clearly  on  the  highest-priority  aspects  of  diabetes 
care,  including  glycemic  control,  foot  exams,  and  eye  exams. 

•  Careful  measurement  of  the  numerators  and  denominators  for 
performance  indicators  will  be  required  to  ensure  effective 
monitoring  of  progress.  Several  measurement  issues  arose  during 
the  demonstration  that  hampered  effective  monitoring — 
specifically  inconsistencies  in  coding  of  foot  exams,  eye  exams, 
and  patient  education,  as  well  as  coding  and  manpower  credit 
for  group  visits.  The  MTFs  also  had  some  difficulty  in  retrieving 
ADS  and  CHCS  laboratory  data. 

•  The  participating  MTFs  educated  both  providers  and  clinic  staff 
on  the  diabetes  practice  guideline,  and  they  involved  clinic  staff 
actively  in  the  implementation  of  new  practices.  This  approach 
seemed  to  pay  off  in  the  willing  participation  by  staff  in  intro¬ 
ducing  new  practices,  including  provision  of  feedback  on  prob¬ 
lems  or  questions  that  arose.  Further,  both  MTFs  reported  they 
were  providing  ongoing  education  for  existing  personnel,  and 
they  were  including  guideline  information  in  the  orientation  ses¬ 
sions  for  incoming  staff  Both  of  these  educational  activities  are 
needed  to  sustain  improved  practices. 

•  Patient  education  is  an  important  aspect  of  diabetes  care,  espe¬ 
cially  for  the  new  diabetes  patient.  Both  MTFs  had  a  strong 
focus  on  patient  education,  but  they  apparently  had  some  prob¬ 
lems  in  effectively  coordinating  specific  aspects  of  the  education, 
which  were  performed  by  different  functions  within  the  MTF. 
Further  assistance  by  MEDCOM  might  be  useful  to  enhance 
the  ability  to  reach  all  patients  and  offer  comprehensive  educa¬ 
tion  for  managing  the  various  aspects  of  their  diabetes. 

•  The  goal  of  actions  to  implement  practice  improvements  is 
sustainability — the  successful  integration  of  new  practices  into 
the  way  MTFs  routinely  “do  business”  for  patient  care.  As  of  the 
end  of  the  demonstration,  the  participating  MTF  teams  had  not 
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yet  reached  the  point  of  fully  maintaining  the  new  practices  they 
introduced.  They  were  well  positioned  to  do  so,  however,  if  they 
could  put  into  place  effective  and  routine  provider  education 
and  performance  monitoring  processes.  The  achievement  of  sus¬ 
tainable  practice  improvements  can  he  encouraged  hy 
MEDCOM  through  ongoing  monitoring  and  technical  support 
for  the  implementation  activities  of  the  Army  MTFs. 

The  MTFs  reported  difficulties  in  exporting  their  achievements 
to  other  teams  and  other  clinics  within  the  MTF.  The  amount 
of  work  and  time  required  to  expand  the  activities  was  an 
important  harrier.  They  found  they  would  have  to  duplicate  on 
a  larger  scale  the  processes  they  had  used  for  one  or  two  clinics 
or  teams,  including  leadership  hy  other  champions,  training  of 
providers  and  clinic  staff,  and  expansion  of  patient  education 
activities.  To  achieve  successful  introduction  and  consolidation 
of  new  hahits  among  a  large  number  of  providers  and  clinic 
staff,  implementation  activities  require  not  only  resources  but 
also  time  to  mature. 

For  patients  with  chronic  conditions,  such  as  asthma  or  diabetes, 
the  establishment  of  a  registry  would  provide  a  centralized 
repository  of  pertinent  data  that  could  be  shared  by  all  MTFs  as 
the  patients  move  around  the  military  system.  Although 
AMEDD  does  not  have  centralized  registries,  many  of  the  local 
MTFs  are  attempting  to  establish  them  for  their  patient  popula¬ 
tions. 

Two  approaches  for  improving  centralized  data  collection  may 
be  considered.  MEDCOM  could  establish  a  centralized  system 
that  collects  the  data  directly  from  automated  data  systems,  per¬ 
forms  analyses  in  the  central  office,  and  generates  trend  reports 
to  the  MTFs.  Alternatively,  the  system  could  use  data  collected 
and  analyzed  locally  by  the  MTFs  and  reported  to  MEDCOM, 
which  then  would  aggregate  the  individual  MTF  results  into 
trend  reports.  The  analyses  we  performed  for  this  report  using 
solely  the  administrative  data  would  be  a  starting  point  for  such 
a  system.  Despite  the  limitations  of  the  information  available 
from  administrative  data,  it  remains  the  most  usable  data  source 
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because  it  comprehensively  covers  a  broad  range  of  services. 
Using  the  data  routinely  is  one  of  the  best  ways  to  stimulate 
improvements  to  coding  and  data  submittals  because  MTFs 
want  their  information  to  be  accurate  if  it  is  used  for  monitor¬ 
ing.  If  the  second  approach  were  chosen,  MEDCOM  would 
need  to  define  consistent  measurement  methods  and  standards 
for  the  MTFs  to  use,  and  it  would  have  to  perform  regular 
audits  for  measurement  consistency  to  ensure  the  integrity  of  the 
data  for  effective  performance  monitoring.  Perhaps  the  most 
important  information  that  emerged  from  the  cost  analysis  was 
that  care  for  diabetes  patients  not  enrolled  at  the  MTFs  repre¬ 
sented  a  substantial  share  of  MTF  costs  for  diabetes  patients, 
especially  for  inpatient  services.  This  finding  raises  the  question 
of  how  the  MTFs  can  best  provide  effective  care  management 
for  these  nonenrollee  patients,  given  that  they  are  episodic  users 
of  the  facilities,  as  reflected  in  their  disproportionate  use  of  inpa¬ 
tient  resources. 


Costs 

With  respect  to  effects  of  improved  diabetes  care  on  MTF  costs  and 
return  on  investment,  it  is  still  too  early  to  document  any  return.  It  is 
necessary  first  to  achieve  observable  changes  in  resource  use  and 
patient  outcomes,  some  of  which  will  require  more  than  one  year  to 
emerge.  MEDCOM  should  continue  to  track  inpatient  use  rates  and 
costs  over  time.  As  cost  information  accumulates,  it  should  be  possi¬ 
ble  to  distinguish  trends  related  to  practice  changes  from  normal  fluc¬ 
tuations  in  health-care  needs  from  year  to  year. 
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Evaluation  Methodology 


Process  Evaluation 

To  capture  the  full  dynamics  of  a  process  as  complex  as  practice 
guideline  implementation,  it  is  important  to  take  into  account  the 
roles  and  interactions  of  the  many  aspects  of  the  system  in  which  the 
guidelines  are  being  implemented.  Figure  A.  1  is  a  diagram  of  rela¬ 
tionships  among  the  different  levels  of  a  health-care  organization 
during  guideline  implementation,  the  stakeholders  involved,  and  the 
dynamics  of  the  implementation  process. 

A  variety  of  stakeholders  need  to  he  considered  to  ensure  that 
individuals  involved  in  implementing  new  practices  anticipate  possi¬ 
ble  impacts  on  the  stakeholders  and  responses  that  might  be  expected 
from  them.  These  groups  include  treatment  program  leadership, 
middle  management,  clinical  and  administrative  staff  working  with 
program  residents,  and  clients  themselves.  The  implementation  team 
itself  consists  of  important  stakeholders  who  not  only  serve  as  team 
members  but  also  have  other  job  responsibilities. 

Information  was  collected  about  the  actions  involved  in  practice 
guideline  implementation  for  participating  MTFs,  the  dynamics  of 
the  change  process,  and  responses  of  participants  to  their  experiences 
with  the  process.  Similarities  and  differences  in  the  attitudes,  motiva¬ 
tions,  and  preferences  of  the  stakeholders  were  considered  as  the 
process  evaluation  information  was  collected  and  results  were  synthe¬ 
sized.  To  capture  changes  in  structures,  processes,  and  issues  as 
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Figure  A.1 

A  System  View  of  Guideline  Implementation 

EXTERNAL  ENVIRONMENT 


RAND/WG277V1.I 


guideline  implementation  moved  forward,  site  visits  were  conducted 
to  collect  information  at  the  baseline  and  at  two  follow-up  times,  as 
shown  in  Table  A.  1 . 

A  participant-observer  approach  was  used  throughout  the 
implementation  process  and  evaluation.  In  addition  to  the  site  visits, 
we  used  routine  progress  reports  and  maintained  an  ongoing  com¬ 
munication  process  to  provide  a  structure  through  which  imple¬ 
menting  MTFs  could  get  assistance  from  each  other,  MEDCOM,  or 
RAND. 

Both  qualitative  and  quantitative  data  collection  methods  were 
used  in  the  process  evaluation  to  collect  information  on  a  set  of  ques¬ 
tions  that  cover  the  dimensions  shown  in  Table  A.l.  Shown  in  Table 
A.2  are  the  specific  topic  areas  covered  and  relevant  data  collection 
methods.  Interviews  and  focus  groups  with  the  implementation  team, 
providers  and  clinic  staff,  quality  management  staff,  and  other  par¬ 
ticipants  yielded  information  on  the  dynamics  of  the  implementation 
process.  Focus  groups  were  conducted  with  three  groups:  the  imple¬ 
mentation  team,  providers,  and  other  clinic  staff.  Participants  in  each 
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Table  A.1 

Dimensions  Addressed  by  the  Process  Evaluation 


Dimension 

Baseline 

Month  3 

Month  9 

Structure  and  organization 

X 

X 

X 

Culture  and  climate 

X 

X 

Current  practices 

X 

X 

X 

Environmental  context 

X 

X 

X 

Stakeholders'  attitudes 

X 

X 

X 

Implementation  plan 

X 

X 

Changes  in  clinic  processes 

X 

X 

AMEDD  support  systems 

X 

X 

Staff  involvement 

X 

X 

Patient  roles  and  reactions 

X 

X 

Monitoring  progress 

X 

X 

Effects  on  stakeholders 

X 

X 

stakeholder  group  were  asked  questions  regarding  their  attitudes 
toward  guideline  implementation,  how  they  worked  with  the  practice 
guideline,  how  they  were  affected  hy  the  implementation  process,  and 
issues  or  concerns  they  identified.  Semistructured  interview  methods 
were  used  for  all  interviews,  group  discussions,  and  focus  groups, 
working  from  lists  of  questions  to  cover  during  each  session. 

A  brief  survey  regarding  stakeholders’  attitudes  toward  practice 
guidelines  and  quality  improvement  processes  was  administered  at  the 
baseline  and  the  final  site  visit.  The  survey  at  the  final  site  visit  also 
included  questions  about  education  received  on  the  guideline,  actions 
taken  to  implement  the  new  practices,  and  effects  of  those  actions  on 
providers  and  clinic  staff 

Documents  and  materials  also  were  important  sources  of  infor¬ 
mation  for  the  process  evaluation.  These  included  written  infor¬ 
mation  about  the  MTF  structure  and  management,  policies  and 
procedures,  data  collection  and  monitoring,  and  materials  developed 
by  the  MTF  implementation  teams  as  they  prepared  and  carried  out 
their  action  plans  to  change  practices.  The  materials  provided  the 
primary  documentation  on  the  actions  planned  by  the  team,  changes 
made  to  clinic  processes,  resulting  events,  and  actions  taken  to  moni¬ 
tor  their  progress. 
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Table  A.2 

Dimensions  Addressed  by  the  Process  Evaluation  Data  Collection  Methods 


Document 

Monitor 

Individual 

Focus 

Culture 

Materials 

Reports 

Interviews® 

Groups 

Survey 

Environmental  Context 

How  supportive  was  culture 
and  climate? 

How  did  culture  and  climate 
change? 

What  were  the  other  factors 
affecting  implementation? 

The  Implementation  Plan 

What  key  guideline  elements 
are  priorities?  X  X 

What  information  is  needed 
to  identify  priorities? 

How  is  guideline  team  orga¬ 
nized?  X 

How  does  guideline  team 
operate? 

How  was  guideline  intro¬ 
duced  to  staff?  X  X 
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Table  A.2 — continued 


Document  Monitor  Individual  Focus  Culture 

Materials  Reports  Interviews^  Groups  Survey 

Were  the  MDs  motivated  to 

adopt  new  practices?  X  X 

What  were  the  effects  of 
changes  on  MDs  and 
responses? 

What  were  the  effects  on 
other  staff  workload, 
demands? 


Roles  and  Reactions  of  Patients 


what  were  patients' 

responses  to  changes  in 
care? 

X 

X 

How  did  the  team  manage 
patient  reactions? 

How  helpful  were  patient 

X 

X 

education  materials? 

What  were  the  effects  on 

X 

X 

physician-patient  relation¬ 
ships? 

X 

X 

Measuring  Implementation  Prog 

ress 

What  were  the  indicators  MTF 

selected  for  monitoring?  X 

X 

What  were  the  MTF  data  sys¬ 
tem  for  monitoring?  X 

What  were  the  monitoring 

X 

X 

X 

lessons  and  actions  taken? 

How  useful  was  monitoring 

X 

X 

X 

to  staff? 

X 

X 

'“Individual  interviews  included  one-on-one 

interviews 

and 

written  questionnaires 

completed  by  key  participants. 


Evaluation  of  Effects  (Outcomes) 

The  evaluation  of  the  effects  of  the  diabetes  practice  guideline  dem¬ 
onstration  was  designed  to  work  entirely  with  administrative  data. 
Ideally,  these  data  would  have  included  a  master  enrollment  fde  for 
beneficiaries  using  the  Army  MTFs  along  with  files  containing  data 
on  health  service  encounters.  Unfortunately,  although  a  master 
TRICARE  enrollment  file  has  centralized  data  on  all  beneficiaries, 
these  data  were  not  available  for  use  by  the  AMEDD.  Therefore,  we 
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had  to  work  entirely  with  data  from  the  health  service  encounters, 
including  SIDR,  SADR,  and  MTF  pharmacy  data  for  MTF  services 
and  HCSR  and  NMOP  data  for  network  provider  services.  An  exten¬ 
sive  process  of  data  extraction,  variable  derivation,  and  diagnostic 
analyses  was  carried  out  to 

•  identify  correcdy  the  diabetes  population  served  by  the  Army 
MTFs  during  the  two-year  study  period; 

•  select  the  study  sample  for  the  analysis  of  guideline  effects  on 
service  delivery,  consisting  of  the  subset  of  the  diabetes  popula¬ 
tion  enrolled  at  any  demonstration  or  control  MTF;  and 

•  establish  a  database  of  all  health-care  encounters  for  the  study 
sample. 

We  first  document  here  the  specific  steps  involved  in  the  data 
extraction  process  and  variable  specification  to  achieve  these  three 
work  products.  Then  we  summarize  the  codes  used  to  define  the  vari¬ 
ables  for  the  analysis  of  demonstration  effects. 

Overview  of  the  Data  Extraction  Process  for  the  Diabetes  Study 

Two  rounds  of  data  extraction  and  file  construction  were  performed 
in  collaboration  with  PASBA  to  establish  the  data  required  for  our 
analyses.  In  round  one,  we  extracted  data  necessary  to  identify  all  the 
diabetic  patients  served  by  any  Army  MTFs  or  health  centers  during 
the  two  study  years,  and  we  established  a  data  file  containing  a  record 
for  each  patient  along  with  descriptive  data  on  them  that  could  be 
obtained  from  the  administrative  data.  In  round  two,  we  extracted 
data  on  all  encounters  for  the  subset  of  patients  enrolled  at  one  of  the 
MTFs  in  the  demonstration  or  control  groups,  which  included  their 
use  of  any  military  or  network  provider  services  for  any  reason. 

We  extracted  the  round-two  data  in  two  steps  because  our  initial 
data  request  to  PASBA  occurred  soon  after  the  end  of  the  last  quarter 
of  the  second  study  year  (January  2001-March  2001).  Data  for  this 
quarter  were  incomplete  for  up  to  six  months  following  the  end  of 
the  quarter  because  of  delays  in  processing  SADR  and  SIDR  data 
about  MTF  encounters,  network  provider  claims  that  become  the 
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HCSRs,  and  NMOP  claims.  Therefore,  the  initial  data  pulls  covered 
only  the  two  calendar  years  of  1999  and  2000.  We  used  these  data  to 
perform  preliminary  analyses  that  were  updated  later  when  data  for 
the  final  quarter  were  more  complete. 

In  addition  to  obtaining  data  from  the  SADR,  SIDR,  HCSR 
outpatient  and  inpatient,  and  NMOP  data  files,  we  extracted  clinical 
chemistry  data  from  the  database  that  was  constructed  through  the 
Triservice  Data  Pull  project  in  which  PASBA  participated.  This  was 
done  only  for  the  patients  in  the  study  sample  that  RAND  identified 
for  round  two.  The  data  obtained  covered  the  period  from  June  2000 
through  June  2001 ,  of  which  only  data  for  June  2000  through  March 
2001  were  relevant  to  service  delivery  during  the  demonstration 
period.  Baseline  laboratory  data  for  the  first  study  year  were  not  avail¬ 
able  because  these  data  had  not  been  accessed  at  the  DoD  level  prior 
to  the  Triservice  Data  Pull  project.  (See  Chapter  Two  for  discussion 
of  our  diagnostic  analysis  of  these  data.) 


Round-One  Data  Extraction  Specifications 

SADR  Data  Files 

Record  extraction  rules:  Keep  all  encounter  records  from  the  SADR 
(MTF  outpatient)  files  that  meet  the  following  criteria: 

•  Encounter  date  between  January  1,  1999,  and  March  31,  2001. 

•  All  individuals  eligible  for  DoD  health-care  benefits,  including 
active-duty  personnel,  dependents,  or  retirees  in  any  service  as 
well  as  all  other  eligible  groups. 

•  Have  a  code  in  any  diagnostic  field  of  250.xx  (diabetes). 

•  Treated  at  any  Army  MTF  or  clinic  (all  Army  Defense  Medical 
Information  System  identifications  (DMIS  IDs),  including 
clinics  and  TMCs). 

The  following  variables  were  extracted  from  the  SADR  file  data: 
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Alternate  care  value  ID  for  MTF  where  enrolled  in 

Appointment  status  type  Prime 

Beneficiary  category  Patient  Zip  code  of  residence 

Calendar  year  (created  by  MEPRS  code  for  clinic  of  ser- 

PASBA)  vice 

ID  for  MTF  treating  the  patient  MTF  location 
Disposition  code  Patient  date  of  birth 

Diagnosis  codes  1  through  4  Patient  gender 

Encounter  date  Sponsor  Social  Security  number 

(SSN) 

Family  member  prefix 

SIDR  Data  Files 

Record  extraction  rules:  Keep  all  encounter  records  from  the  SIDR 

(MTF  inpatient)  files  that  meet  the  following  criteria: 

•  Admission  date  between  January  1,  1999,  and  March  31,  2001. 

•  All  individuals  eligible  for  DoD  health-care  benefits,  including 
active-duty  personnel,  dependents,  or  retirees  in  any  service  as 
well  as  all  other  eligible  groups. 

•  Have  a  code  in  any  diagnostic  field  of  250.xx  (diabetes). 

•  Treated  at  any  Army  MTF  (all  Army  DMIS  IDs,  including 
clinics  and  TMCs). 


The  following  variables  were  extracted  from  the  SIDR  file  data: 


Alternate  care  value 
Date  of  admission 
Admission  source 
Beneficiary  category 
Calendar  year  (created  by 
PASBA) 

Date  of  disposition  (discharge) 
Type  of  disposition 
ID  of  MTF  of  service 
MTF  name 

MEPRS  code  for  inpatient  unit 


Diagnosis  codes  1  through  8 
Date  of  disposition  (discharge) 
Patient  Zip  code  of  residence 
MTF  location 
Number  of  diagnoses 
Patient  date  of  birth 
Patient  gender 
Sponsor  SSN 
Family  member  prefix 
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MTF  Pharmacy  Files 

Record  extraction  rules:  Keep  all  records  from  the  MTF  pharmacy  files 
that  meet  the  following  criteria: 


•  Beginning  date  of  service  between  January  1,  1999,  and  March 
31,2001. 

•  All  individuals  eligible  for  DoD  health-care  benefits,  including 
active-duty  personnel,  dependents,  or  retirees  in  any  service  as 
well  as  all  other  eligible  groups. 

•  At  least  one  prescription  from  the  list  of  diabetes  medications 
(listed  below)  using  Therapeutic  Class  Code  and  National  Drug 
code  that  was  provided  by  the  PEC. 

•  Filled  a  prescription  at  any  Army  MTF  pharmacy  (all  Army 
DMIS  IDs,  including  clinics  and  TMCs). 

Outpatient  Network  Provider  (HCSR)  Files 

Record  extraction  rules:  Keep  all  encounter  records  from  the  HCSR 
outpatient  files  that  meet  the  following  criteria: 


•  Beginning  date  of  service  between  January  1,  1999,  and  March 
31,2001. 

•  All  individuals  eligible  for  DoD  health-care  benefits,  including 
active-duty  personnel,  dependents,  or  retirees  in  any  service  as 
well  as  all  other  eligible  groups. 

•  Have  a  code  in  any  diagnostic  field  of  250.xx  (diabetes). 

•  Reside  in  the  catchment  area  of  an  Army  MTF  or  clinic.  Keep 
all  records  coded  equal  to  any  of  the  Army  facility  parent  DMIS 
IDs,  including  health  centers  and  MTFs.  This  will  pick  up  peo¬ 
ple  residing  in  the  parent  catchment  area  even  if  they  used  a 
freestanding  clinic  with  a  separate  DMIS  ID. 

The  following  variables  were  extracted  from  the  HCSR  Outpa¬ 
tient  file  data: 
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Beginning  date  of  service 
Beneficiary  category 
Patient  catchment  area 
Calendar  year 

Defense  Enrollment  Eligibility 
Reporting  System  (DEERS) 
dependent  suffix  (DDS) 
Diagnoses  1  through  4 
Date  of  birth 
End  date  of  service 
Enrollment  DMIS  ID 


Enrollment  DMIS  parent 
region 

Enrollment  status 
Health  service  region 
Patient  age 
Patient  gender 

Patient  Zip  code  of  residence 
Primary  diagnosis 
Sponsor  SSN 
Sponsor  service  branch 


Another  variable  we  did  not  request  was  the  “type  of  service” 
variable,  which  identifies  the  specific  type  of  outpatient  visit  for 
which  the  HCSR  claim  was  submitted.  This  would  have  been  used  to 
identify  visits  to  network  provider  ERs,  which  should  have  been 
included  the  indicator  for  ER  use  rates.  This  variable  should  be 
included  in  any  future  database  used  for  monitoring  this  indicator. 

Inpatient  Network  Provider  (HCSR)  Files 

Record  extraction  rules:  Keep  all  encounter  records  from  the  HCSR 
Inpatient  files  that  meet  the  following  criteria: 

•  Beginning  date  of  service  between  January  1,  1999,  and  March 
31,2001. 

•  All  individuals  eligible  for  DoD  health-care  benefits,  including 
active-duty  personnel,  dependents,  or  retirees  in  any  service  as 
well  as  all  other  eligible  groups. 

•  Have  a  code  in  any  diagnostic  field  of  250.xx  (diabetes). 

•  Reside  in  the  catchment  area  of  an  Army  MTE  or  health  center. 
Keep  all  records  coded  equal  to  any  of  the  Army  facility  parent 
DMIS  IDs,  including  health  centers  and  MTEs.  This  will  pick 
up  people  residing  in  the  parent  catchment  area  even  if  they 
used  a  freestanding  clinic  with  a  separate  DMIS  ID. 

The  following  variables  were  extracted  from  the  HCSR  Inpa¬ 
tient  file  data: 
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Admission  date 
Beginning  date  of  service 
Beneficiary  category 
Patient  catchment  area 
Calendar  year 
DDS 

Discharge  status  code 
Primary  diagnosis 
Diagnoses  1  through  8 
Enrollment  DMIS  parent 
region 


Date  of  hirth 
End  date  of  service 
Enrollment  DMIS  ID 
Enrollment  status 
Patient  age 
Patient  gender 

Patient  Zip  code  of  residence 
Sponsor  SSN 
Health  service  region 
Sponsor  service  branch 


NMOP  Files 

Record  extraction  rules:  Keep  all  records  from  the  NMOP  files  that 
meet  the  following  criteria: 

•  Beginning  date  of  service  between  January  1,  1999,  and  March 
31,2001. 

•  All  individuals  eligible  for  DoD  health-care  benefits,  including 
active-duty  personnel,  dependents,  or  retirees  in  any  service  as 
well  as  all  other  eligible  groups. 

•  At  least  one  prescription  from  the  list  of  diabetes  medications 
(listed  below)  using  Therapeutic  Class  Code  and  National  Drug 
code  that  was  provided  by  the  PEC.  Eirst  pull  records  using 
Therapeutic  Class  Code  and  then  pull  by  National  Drug  Code 
from  that  subset  of  records. 


The  following  variables  were  extracted  from  the  NMOP  file 

data: 


Alternate  care  value 
Clinic  Zip  code  +  4 
Calendar  year 
DDS 

Enrollment  DMIS  ID 
Enrollment  parent  DMIS  ID 


Patient  date  of  birth 
Patient  Zip  code  of  residence 
Prescription  transaction  date 
(date  filled,  not  date  posted) 
TRICARE  region  code 
Sponsor  service  branch 
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Gender 

National  drug  code 


List  of  Diabetes  Medications 

Sulfonylureas  (first  generation) 
Chlorpropamide  [Diabenese®] 
Tolazamide  [Tolinase®] 
Tolbutamide  [Orinase®] 
Acetohexamide  [Dymelor®] 
(second  generation) 

Glyburide  [Diabeta®,  Micro- 
nase®] 

Glyburide  micronized  [Glynase®] 
Glipizide  [Glucotrol®] 

Glipizide  extended  release  [Glu¬ 
cotrol  XL®] 

Glimerperide  [Amaryl®] 
Biguanides 

Metformin  [Glucophage®] 
Glyburide/ metformin  [Gluco- 
vance®] 

Alpha-Glucosidase  Inhibitors 
Acarbose  [Precose®] 

Miglitol  [Glyset®] 
Thiazolidinediones 
Rosiglitazone  [Avandia®] 
Pioglitazone  [ActosTM®] 
Troglitazone  or  [Rezulin®] 
Meglitinides 
Repaglinide  [Prandin®] 
Nateglinide  [Starlix®] 


Sponsor  SSN 
Therapeutic  class  code 


Insulins 

Insulin  (human  or  beef/pork) 
Insulin  lispro  [Humalog®] 
Insulin  glargine  [Lantis®] 
Protamine  zinc  insulin  (PZI) 
Novo  Nordisk  insulins: 
Novolin-N  for  NPH 
Novolin-R  for  Regular, 
Novolin-L  for  Lente 
Novolin-U  for  Ultralente 
Novolin  70/30  for  premixed 
insulin 

Eli  Lilly  insulins: 
Humubn®-N  for  NPH 
Humulin®-R  for  Regular 
Humulin®-L  for  Lente 
Humulin®70/30  for  pre¬ 
mixed  insulin 
Glucose  test  strips 
Precision  QID 
One  Touch 
Accucheck  Advantage 
Chemstrip  BG 
Dextrostix 
Diascan 
Glucofilm 

Glucometer  Encore,  Elite 
Eirst  Choice 


Round-Two  Data  Extraction  Specifications 

Two  steps  were  involved  in  the  final  data  extraction  that  was  con¬ 
ducted  during  round  two.  first,  we  created  the  unique  identifier  for 
all  individual  diabetic  patients  using  the  round-one  data.  Then  we 
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extracted  data  on  all  encounters  for  the  subset  of  parients  who  were 
always  enrolled  at  one  of  the  demonstration  or  control  MTFs  during 
each  of  the  two  study  years. 

Creation  of  the  RAND  ID 

A  single  RAND  ID  was  created  for  each  person  that  corrected  the 
identifiers  for  people  with  incorrect  or  multiple  Family  Member  Pre¬ 
fix  (FMP)  relationship  variables  in  the  encounter  and  claims  data. 
This  identifier  consisted  of  the  sponsor’s  SSN  plus  a  two-character 
relationship  identifier  selected  based  on  the  data  available  to  us  in  the 
encounter  and  claims  data  we  used  to  identify  our  patient  population. 
For  each  encounter  or  claim  record,  we  identified  the  combination  of 
the  sponsor  SSN  and  the  date  of  birth  of  the  individual  receiving  the 
service,  which  were  the  basis  for  establishing  the  RAND  identifiers. 

The  unique  RAND  ID  was  established  in  a  two-step  process. 
We  first  created  two  files  that  identified  all  DDS  and  FMP  codes  for 
each  unique  SSN/patient  birth  date  combination.  The  first  file  con¬ 
tained  all  of  the  DDS  codes  reported  in  the  network  provider  and 
NMOP  claims  files  for  each  SSN/patient  birth  date  combination,  as 
well  as  counts  of  the  total  records  with  each  DDS.  The  other  file  con¬ 
tained  all  of  the  FMP  codes  in  the  SIDR,  SADR,  and  MTF  phar¬ 
macy  records  for  each  SSN/patient  birth  date  combination,  as  well  as 
counts  of  the  total  records  with  each  FMP.  Then  we  combined  the 
information  from  these  two  files,  generating  a  single  relationship  vari¬ 
able  for  each  sponsor  SSN/patient  birth  date  in  the  encounter  and 
claims  data. 

Combining  the  information  from  these  two  files,  a  single  rela¬ 
tionship  variable  was  generated  for  each  sponsor  SSN/patient  birth 
date  in  the  encounter  and  claims  data.  The  RAND  ID  variable  was 
created  by  combining  the  sponsor  SSN  and  the  new  relationship  vari¬ 
able.  Precedence  was  given  to  the  DDS  code  because  this  is  supposed 
to  be  consistent  throughout  the  system,  whereas  the  FMP  code  is 
only  consistent  within  the  facility: 

•  if  there  was  only  one  DDS,  the  relationship  variable  was  set  to 
that  DDS; 
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•  Otherwise,  the  variable  was  set  to  the  DDS  with  highest  numher 
of  occurrences; 

•  if  there  was  no  DDS,  the  variable  was  set  to  the  FMP  with  high¬ 
est  number  of  occurrences. 

This  coding  process  changed  the  relationship  code  for  an  esti¬ 
mated  1.1  percent  of  the  patient  population.  For  less  than  0.75  per¬ 
cent  of  the  cases,  multiple  SSN/birth  date  combinations  received  the 
same  relationship  variable  assignment.  Most  of  these  were  cases  for 
which  the  date  of  birth  was  entered  incorrecdy  (e.g.,  month  and  day 
switched),  which  we  corrected.  Some  were  real  errors  in  assignments, 
which  were  not  corrected. 

When  we  created  the  unique  RAND  ID  variable  to  identify 
patients  correctly  in  the  database,  we  appended  this  variable  to  each 
of  the  encounter  or  claims  data  records  so  that  data  in  the  master  file 
could  be  linked  as  accurately  as  possible  to  service-use  data  for  each 
patient.  Using  these  data,  we  created  a  single  summary  file  that  cap¬ 
tured  most  of  the  relevant  information  from  the  Phase  1  encounter 
and  claims  data. 

Creation  of  Analysis  Master  Files 

Two  summary  files  containing  patient  demographics  and  other  char¬ 
acteristics  were  created  using  the  Phase  1  encounter  data.  One  file 
contains  one  record  per  person  per  study  year;  the  other  file  contains 
one  record  per  person.  Summary  variables  were  derived  by  coding  for 
each  variable  on  individual  encounter  or  claim  records  and  then 
summarizing  them  at  the  person  level  based  on  the  unique  RAND  ID 
codes.  For  the  first  step,  the  following  variables  were  created  for  each 
claim  record. 

Study  Year 

Demo-control 


Study  Year  of  April  1  to  March  3 1 ,  where  year  =  0  is 
for  1999-2000  and  year  =  1  for  2000-2001 

Demo  =  any  claim  is  for  service  in  a  demonstration 
MTF  or  indicates  that  the  person  is  enrolled  in  a 
demonstration  site 

Control  =  any  claim  is  for  service  in  a  control  site  or 
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indicates  that  the  person  is  enrolled  in  a  control 
site 


Other  =  no  claims  for  service  in  either  a  demonstra¬ 
tion  or  control  site 

Note:  For  network  provider  claims,  this  variable  is 
identified  based  on  the  MTF  catchment  area  vari¬ 
able  or  by  a  Zip  code  matched  to  an  Army  DMIS 
ID 


Active-duty 

Army 

Enrollment  ID 


Identifies  active-duty  personnel  based  on  beneficiary 
category  variable 

Identifies  Army  personnel  based  on  beneficiary  cate¬ 
gory  variable 

Identifies  whether  person  enrolled  to  network, 

MTF,  or  not  enrolled  based  on  enrollment  DMIS 
ID 


Then  the  variables  were  summarized  by  person  and  study  year. 
Each  data  source  (e.g.,  SIDR,  SADR,  PEC)  was  summarized  by 
person/year  and  data  in  these  summary  files  were  combined.  In  addi¬ 
tion  to  the  variables  described  above,  place  of  service  and  enrollment 
summary  variables  are  created.  Finally,  patient  demographic  variables 
for  age  and  family  relationship  were  created.  The  variables  were 
defined  using  the  following  coding. 


DMIS  IDs  for  DMIS  IDs  are  the  unique  identifier  codes 
MTF  location  of  assigned  to  each  MTF  or  Army  health  center, 
service  An  array  of  all  DMIS  IDs  for  MTFs  that  were 

locations  of  service  for  the  patient’s  MTF 
encounters  during  the  study  year. 


DMIS  IDs  for 
TRICARE 
Prime  enroll¬ 
ments 


An  array  of  all  DMIS  IDs  for  MTFs  where  the 
patient  was  enrolled,  as  recorded  in  the  patient’s 
MTF  encounters  during  the  study  year. 


User  population  Takes  one  of  the  following  values  for  location  of 
service  for  the  study  year.  Based  on  the  location 
of  service  DMIS  ID  for  each  MTF  claim. 
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Enrolled  popula¬ 
tion 


Always  Same  =  all  nonmissing  DMIS  IDs  for 
study  year  are  the  same  and,  for  the  second  year, 
also  are  the  same  as  the  last  DMIS  ID  for  the 
fourth  quarter  of  the  previous  year. 

Same  for  SY,  No  Match  with  Previous  SY  =  all 
nonmissing  DMIS  IDs  for  the  study  year  are 
the  same  hut  not  the  same  as  last  DMIS  ID  for 
the  fourth  quarter  of  the  previous  year. 

Same  for  SY,  No  DMIS  Previous  SY  =  all  non¬ 
missing  DMIS  IDs  for  the  study  year  are  the 
same  and  no  DMIS  ID  for  the  fourth  quarter  of 
the  previous  year. 

Not  the  Same  =  nonmissing  DMIS  IDs  are  not 
the  same  for  the  study  year. 

No  Location  Information  =  any  other  conditions. 

Takes  one  of  the  following  values  for  enrollment 
information  for  the  study  year.  Based  on  the 
enrollment  DMIS  ID  for  each  MTF  or  HCSR 
claim. 

Always  Same  =  all  nonmissing  enrollment  DMIS 
IDs  for  study  year  are  the  same  and,  for  the  sec¬ 
ond  year,  also  are  the  same  as  the  last  DMIS  ID 
for  the  fourth  quarter  of  the  previous  year. 

Same  for  SY,  No  Match  with  Previous  SY  =  all 
nonmissing  enrollment  DMIS  IDs  for  the  study 
year  are  the  same  hut  not  the  same  as  last  the 
DMIS  ID  for  fourth  quarter  of  the  previous 
study  year. 

Same  for  SY,  No  DMIS  Previous  SY  =  all  non¬ 
missing  enrollment  DMIS  IDs  for  the  study 
year  are  the  same  and  no  DMIS  ID  for  the 
fourth  quarter  of  the  previous  year. 

Enrolled,  Not  the  Same  =  nonmissing  enrollment 
DMIS  IDs  exist  for  the  study  year  hut  are  not 
the  same. 

PEC  Claims  Only  for  Year  =  MTF  pharmacy 
records  exist  hut  no  enrollment  DMIS  IDs  on 
service-use  records  for  the  study  year. 

Never  Enrolled  in  Year  =  neither  any  enrollment 
DMIS  IDs  on  service-use  records  nor  any  MTF 
pharmacy  records  for  the  study  year. 
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Type  of  TRI¬ 
CARE  Prime 
enrollment 


Analysis  popula¬ 
tion 


Takes  one  of  the  following  values  for  type  of 
enrollment  for  the  study  year: 

Enrolled  to  Demo  =  enrollment  DMIS  ID  is  ever 
a  demo  site  in  study  year. 

Enrolled  to  Control  =  enrollment  DMIS  ID  is 
ever  a  control  site  in  study  year. 

Enrolled  to  Network:  Demo  =  enrollment  DMIS 
ID  is  ever  network,  patient  Zip  code  in  demo 
catchment  area,  and  place  of  service  DMIS  ID 
is  ever  a  demo  site. 

Enrolled  to  Network:  Control  =  enrollment 
DMIS  ID  is  ever  network,  patient  Zip  code  in 
control  catchment  area,  and  place  of  service 
DMIS  ID  is  ever  a  control  site. 

Enrolled  to  Network:  Other  =  last  enrollment 
DMIS  ID  for  study  year  is  network  in  demo  or 
control  catchment  area  and  place  of  service 
DMIS  ID  is  never  a  demo  or  control  site. 

Enrolled  to  Other  MTE  =  last  enrollment  DMIS 
ID  for  study  year  is  not  missing  and  is  not 
within  networks  of  interest. 

Never  Enrolled  in  Year  =  enrollment  DMIS  ID  is 
missing  for  entire  study  year  and  no  MTE 
pharmacy  claims 

PEC  Claims  Only  for  Year  =  MTE  pharmacy 
records  exist  hut  no  enrollment  DMIS  IDs  on 
service-use  records  for  the  study  year. 

Indicates  whether  person  is  in  the  analysis  popu¬ 
lation  for  the  study  year.  Based  on  enrolled 
population  (enrpop) 

1  =  If  “enrolled  population”  variable  is  “Always 
Same”  or  “Same  for  SY,  No  DMIS  prev  SY” 
and  the  first  enrollment  DMIS  ID  is  a  demon¬ 
stration  site 

2  =  if  “enrolled  population”  variable  is  “Always 
the  Same”  or  “Same  for  SY,  No  DMIS  prev  SY” 
and  the  first  enrollment  DMIS  ID  is  a  control 
site. 

0  =  Otherwise. 
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Age  for  study  year 

Five-year  age 
categories 

Relationship  cate¬ 
gory 


Age  at  December  3 1 ,  calculated  from  date  of 
birth  and  study  year 

Four-level  age  category  variable  for  study  year: 

Less  than  18  years,  18-44  years,  45-64  years,  or 
65+  years 

Classified  as  child,  spouse,  parent,  or  other 


Creation  of  Identifier  Files  for  Data  Extraction  from  DDS 
and  FMP  Files 

For  the  round-two  data  extraction  process,  two  files  were  sent  to 
PASBA  that  contained  lists  of  unique  patient  identifiers  for  all  the 
diabetic  patients  we  identified  as  being  in  our  study  sample  for  the 
analysis  of  guideline  effects.  We  requested  an  extraction  of  all 
encounter  records  for  these  individuals  for  all  MTF  services,  network 
provider  services,  and  pharmacy  prescriptions.  One  file  was  to  be 
used  to  extract  all  claims  for  each  patient  identifier  from  files  that  use 
the  DDS  relationship  variable  (outpatient  and  inpatient  network  pro¬ 
vider  files,  NMOP  files),  and  the  other  was  to  be  used  to  extract 
claims  from  files  that  use  the  FMP  identifier  (SIDR,  SADR,  PEC 
pharmacy  data) . 

To  create  the  identifier  files,  an  index  file  was  constructed  that 
contained  all  unique  sponsor  SSN/FMP  and  sponsor  SSN/DDS 
combinations  for  all  patients  identified  as  always  enrolled  at  one  of 
the  demonstration  or  control  sites.  All  identifiers  reported  in  all  the 
MTF  encounter  records  and  network  provider  HCSRs  were  captured. 
Patient  identifier  records  then  were  output  to  each  of  the  two  identi¬ 
fier  files,  using  the  following  rules: 

A.  Identifier  file  for  extraction  of  data  from  files  using  the  DDS  rela¬ 
tionship  variable  (network  provider  NCSRs,  NMOP) — 

1.  If  the  relationship  variable  is  based  on  a  DDS,  then  output  the 

record  with  sponsor  SSN  and  DDS. 


Evaluation  Methodology  121 


2.  If  the  relationship  variable  is  based  on  FMP: 

a.  If  the  person  is  a  child  (FMP  1-19)  then  do  the  following 

1)  If  FMP  is  1-4  then  output  one  record  for  each  FMP  1-4 
(four  records) 

2)  If  FMP  is  5-19  then  output  record  with  this  FMP  as  well 
as  one  record  for  each  FMP  1  -4  (five  records) 

b.  If  the  person  is  a  spouse  (FMP  30-39): 

1)  If  FMP  is  30  or  31  then  output  one  record  for  both  FMP 
30  and  31  (two  records) 

2)  If  FMP  is  32-39  then  output  record  with  this  FMP  as  well 
as  one  record  for  each  FMP  30  and  31  (three  records). 

c.  If  the  person  has  any  other  code,  output  one  record. 

B.  For  identifier  files  for  extraction  of  data  from  files  using  the  FMP 
relationship  variable  (SIDR,  SADR,  MTF  pharmacy),  only  the  FMP 
variable  is  used  because  there  is  no  DDS  identifier  in  the  encounter 
records — 

a.  If  the  person  is  a  child  (FMP  1-19)  then  do  the  following: 

1)  If  FMP  is  1-4  then  output  one  record  for  each  FMP  1-4 
(four  records) 

2)  If  FMP  is  5-19  then  output  record  with  this  FMP  as  well 
as  one  record  for  each  FMP  1-4  (five  records). 

b.  If  the  person  is  a  spouse  (FMP  30-39)  then  do  the  following: 

1)  If  FMP  is  30  or  31  then  output  one  record  for  both  FMP 
30  and  31  (two  records) 

2)  If  FMP  is  32-39  then  output  record  with  this  FMP  as  well 
as  one  record  for  each  FMP  30  and  31  (three  records) 

c.  If  the  person  has  any  other  code,  output  one  record. 

Definition  of  Key  Outcome  Variables 

Indicators  selected  for  the  evaluation  of  guideline  effects  were  those 
that  could  be  measured  using  available  administrative  data  on  health¬ 
care  encounters,  use  of  prescription  medications,  and  the  demo¬ 
graphic  and  clinical  characteristics  of  the  clients.  The  data  used  to 
derive  these  variables  were  in  the  master  file  on  the  diabetic  patients 
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in  the  study  sample,  along  with  the  comprehensive  encounter  data 
obtained  in  the  round-two  data  extraction.  For  each  outcome  vari- 
ahle,  we  subset  the  records  for  the  study  sample  from  the  master  file 
and  then  used  the  unique  RAND  ID  codes  to  merge  the  patient  data 
to  the  round-two  encounter  or  claims  data  required  to  derive  the  vari¬ 
able.  When  we  obtained  the  round-two  data  from  PASBA,  we 
appended  the  RAND  ID  codes  to  each  encounter  or  claim  record  in 
the  data  using  the  sponsor  SSN/patient  birth  date  combination  as  the 
linking  variable. 

We  were  able  to  obtain  the  data  for  all  encounters  from  the 
round-two  data  extraction  except  the  network  provider  outpatient 
services.  The  DoD  files  containing  these  records  were  extremely  large 
because  they  contained  records  for  all  TRICARE  beneficiaries. 
Because  of  the  limitations  of  the  data  system  that  PASBA  used  to 
extract  the  data,  they  could  not  use  standard  data  management  meth¬ 
ods  to  merge  the  index  file  of  patient  identifiers  to  the  identifiers  on 
the  claims  data.  They  attempted  to  perform  the  data  abstraction  on 
subsets  of  the  DoD  data  by  month,  but  the  resulting  data  files  were 
inconsistent  in  format  and  content  and  we  could  not  use  them. 

The  specific  data  elements  or  codes  used  the  define  these  vari¬ 
ables  are  listed  below: 


Indicator  of  Guide¬ 
line  Effect  Codes  Used  for  the  Definition 


Number  of  primary 
care  visits  per  dia¬ 
betic  patient: 

All  patients 
Patients  on  insulin 
Non-insulin  users 


Percentage  of 
non-insulin 
dependent  dia¬ 
betic  patients 
using  medications 
to  control  hyper¬ 
glycemia 


Primary  care  visit  was  defined  as  a  visit  to  an  MTF  internal 
medicine  (BAA),  family  practice  (BGA),  pediatric  (BDA), 
adolescent  (BDB),  primary  care  (BHA),  or  flight  medicine 
(BJA)  clinics  (MEPRS  codes). 

Patient  on  insulin  was  defined  as  one  with  at  least  two  insu¬ 
lin  prescriptions,  including  Humalog,  Humulin,  lletins,  Iso- 
phane,  Insulatard,  Lantis,  Mixtard,  Novolin,  Relion,  human 
insulin,  beef  insulin,  pork  insulin.  Protamine  zinc,  or  Velo- 
sulin.  All  others  were  coded  as  non-insulin  users. 

Patient  use  of  oral  hypoglycemic  agents  was  defined  as  hav¬ 
ing  at  least  one  prescription  filled  for  one  of  the  sulfonyl- 
ureas,  biquanides,  thiazolidinediones,  or  meglitinides. 
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Percentage  of  dia¬ 
betic  patients 
who  had  at  least 
one  eye  exam 
annually 

Number  of  ER  visits 
per  diabetic 
patient 


Eye  exam  was  defined  as  a  visit  to  an  MTF  optometry  clinic 
(BHC)  or  ophthalmology  clinic  (BBD)  (MEPRS  codes). 


MTF  ER  visit  was  identified  using  MEPRS  code  Bl  (network 
provider  ER  use  data  were  not  available  for  this  study,  but 
also  should  be  included). 


Number  of  hospi-  Each  SIDR  or  network  provider  inpatient  encounter  was 
talizations  per  identified  as  a  hospitalization, 

diabetic  patient 


Cost  Estimation  Methodology 

Estimation  of  MTF  Unit  Costs 

To  estimate  the  costs  of  care  for  diabetic  patients  at  the  Army  MTFs 
included  in  this  study,  MEPRS  financial  data  were  used  to  develop 
sets  of  unit  costs  for  inpatient  and  outpatient  encounters.  The  rele¬ 
vant  estimated  unit  cost  then  was  applied  to  each  unit  of  service 
included  in  the  SIDR  and  SADR  encounter  records.  We  note  there 
has  been  some  criticism  within  the  DoD  that  the  MEPRS  data  over¬ 
estimates  the  MTEs’  costs  of  doing  business.  The  source  of  this  criti¬ 
cism  is  a  reported  over-estimation  of  the  available  hours  of  military 
personnel  time  for  patient  care  activities  because  personnel  often  do 
not  record  time  that  they  spend  on  military-related  activities.  While 
acknowledging  this  issue,  we  also  understand  that  MEPRS  offers  the 
best  available  data,  and  it  is  the  basis  for  all  other  cost  estimations  for 
the  demonstration. 

The  cost  estimation  methodology  we  developed  mirrors  its 
approach  the  PECA  method  developed  by  SRA  for  the  Medicare- 
DoD  Subvention  Demonstration.  Eor  this  cost  analysis,  we  used  cost 
and  workload  data  that  SRA  generated  for  MTE  outpatient  clinics  or 
inpatient  wards  for  all  MTEs  in  the  DoD  system  for  EY  1998.  The 
estimated  unit  costs  included  total  direct  and  indirect  expenses  for 
each  MTE  cost  center  (ward  or  clinic),  including  direct  expenses  for 
staff  time  and  supplies  as  well  as  indirect  expenses  for  ancillary  clinical 
services,  administrative  services,  and  maintenance  and  other  support 
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services.  We  summarize  here  the  methodology  for  calculating  the 
inpatient  and  outpatient  costs. 

We  updated  the  unit  costs  to  FY  1999  estimates  hy  applying  an 
inflation  factor  of  1.4  percent.  These  same  unit  costs  were  applied  to 
encounters  for  both  study  years.  By  holding  costs  constant  over  time, 
any  observed  changes  in  costs  between  study  years  one  and  two  can 
be  attributed  to  changes  in  utilization. 

We  tested  two  references  for  Medicare  cost  increases  to  deter¬ 
mine  the  1.4  percent  inflation  rate.  The  first  was  the  trend  in  the 
U.S.  per-capita  costs  (USPCC)  for  fee-for-service  beneficiaries  that 
the  HCFA  Office  of  the  Actuary  calculates  each  year.  For  the  years 
1996  through  1999,  the  USPCC  increased  at  an  annual  rate  of  1.4 
percent.  We  also  used  the  annual  rate  of  increase  in  the  M+C  county- 
level  capitation  rates,  which  the  Balanced  Budget  Act  of  1997  man¬ 
dated  are  to  be  equal  to  the  rate  of  increase  in  Medicare  fee-for- 
service  costs.  The  annual  updates  used  by  HCFA  to  establish  the 
capitation  rates  for  calendar  years  1999  and  2000  were  1.88  percent 
and  0.90  percent  increases,  respectively,  over  the  previous  year.  These 
also  average  to  1.4  percent.  Because  DoD  payment  policies  mirror 
Medicare  policies,  payments  discounted  using  this  inflation  rate  rep¬ 
resent  increases  in  what  either  DoD  or  Medicare  would  have  paid 
community  providers  if  the  service  had  been  provided  in  FY  1998 
instead  of  FY  1999. 

Inpatient  Stays 

We  estimated  the  cost  per  inpatient  stay  for  each  MTF  inpatient  stay 
using  the  following  formula: 

Cost  for  inpatient  stay  i  in  wardy  = 

(medical  per  diem  cost)  x  (number  of  days)^^  + 

(surgical  per  diem  cost)  -  x  (number  of  days)  -  + 
surgical  cost  for  surgical  DRG, 

where  the  number  of  bed  days  for  each  type  of  inpatient  ward — 
medical  or  surgical — is  the  sum  of  the  ward  and  intensive-care  unit 
(ICU)  days  in  the  SIDR.  DRG  is  the  Diagnosis-Related  Group 
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assigned  to  each  inpatient  stay  based  on  the  patient’s  principal  diag¬ 
nosis  and  treatment.  Medicare  uses  DRGs  as  the  basis  for  payments 
for  inpatient  services,  and  DoD  uses  DRGs  to  establish  amounts 
billed  to  third-party  insurers  for  MTF  inpatient  services. 

For  each  inpatient  ward  in  an  MTF  identified  by  the  MEPRS 
level-3  accounts  (the  level  that  inpatient  wards  are  coded  in  the 
SIDR),  we  obtained  the  following  MEPRS  data  that  we  used  to  cal¬ 
culate  average  total  per-diem  expenses: 

a.  Total  expenses  including  all  stepped-down  expenses  from 

MEPRS  accounts  D  and  E  surgical  expenses  (anesthe¬ 

sia,  surgery  suite,  and  recovery-room  expenses).*  These  costs 
included  clinical  salaries,  direct  operating  costs,  support  costs, 
allocated  IGU  and  ancillary  service  costs,  allocated  costs  from 
purification  of  cost  pools  that  contain  costs  related  to  more  than 
one  account,  and  resource-sharing  costs  that  SRA  assigned  to  the 
inpatient  ward. 

b.  Total  number  of  occupied  bed  days  (OBD)  during  the  year, 
which  will  be  used  with  total  expenses  to  generate  an  estimated 
total  expense  per  OBD. 

c.  Eor  each  surgical  DRG,  we  obtained  an  estimated  average  MTE- 
level  surgical  expense  that  included  expenses  for  anesthesia,  sur¬ 
gery  suite,  and  recovery  room.  This  cost  estimate  was  derived  as 
the  total  MTE  surgical  expenses  divided  by  the  total  weights  of 
surgical  DRGs  during  the  year,  where  surgical  costs  were  esti¬ 
mated  using  the  same  method  that  SRA  applied  for  the  PEGA 
calculations.  Eor  each  surgical  disposition,  we  multiplied  the 
MTE  average  surgical  cost  by  the  DRG  weight  for  the  DRG 
assigned  to  the  patient  stay. 

This  approach  allowed  us  to  capture  all  expenses  for  an  inpatient 
stay  using  a  consistent  methodology  across  all  the  years  of  inpatient 


*  The  MEPRS  D  accounts  are  clinical  ancillary  services  (e.g.,  pharmacy,  pathology,  intensive 
care),  and  the  E  accounts  are  support  services  (e.g.,  administration,  housekeeping,  laundry, 
depreciation). 
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records  included  in  our  analysis.  This  method  smoothes  out  errors  in 
reporting  movement  of  patients  between  ICUs  and  regular  inpatient 
wards  hy  estimating  average  per-diem  costs  that  include  costs  for  the 
regular  ward  services  plus  related  ICU  services.  At  the  same  time,  it 
captures  the  onetime  costs  associated  with  the  surgical  procedure  per¬ 
formed  for  each  surgical  stay  hy  applying  these  costs  separately  for 
each  event.  The  method  also  allows  costs  to  increase  with  length  of 
stay,  thereby  capturing  some  of  the  additional  costs  incurred  by  the 
older  population.  However,  this  approach  assumes  that  ancillary  costs 
are  a  linear  function  of  days,  whereas  it  is  known  that  these  costs  tend 
to  be  concentrated  in  the  early  days  of  an  inpatient  stay  (Carter  and 
Melnick,  1990).  Therefore,  the  method  sacrifices  some  precision  in 
estimating  ancillary  service  costs,  although  SRA  has  informed  us  that 
total  MTF  ancillary  costs  correlate  strongly  with  length  of  stay. 

Outpatient  Visits 

For  each  clinic  in  an  MTF  identified  by  the  MEPRS  level-four 
accounts  (the  level  that  clinics  are  coded  in  the  SADR),  we  obtained 
the  following  MEPRS  data  that  we  used  to  calculate  average  total 
expenses  per  outpatient  visit: 

a.  Total  MEPRS  level-four  expenses  for  the  clinic  for  each  year, 
including  the  resource-sharing  expenses  that  SRA  has  estimated 
and  assigned  to  each  clinic. 

b.  The  MEPRS  count  of  total  outpatient  visits  in  the  clinic  during 
the  year. 

c.  Within  the  total  expenses,  separate  identification  of  the  expenses 
for  laboratory,  radiology,  pharmacy,  all  other  ancillary  services 
(including  allocated  costs  from  purification  of  cost  pools),  and 
resource  sharing. 

These  data  allowed  us  to  calculate  the  average  total  cost  per  visit 
for  each  clinic  in  an  MTE  and  to  estimate  the  shares  of  the  total  clinic 
expenses  that  are  attributable  to  ancillary  services. 


APPENDIX  B 

Analyses  of  Diabetes  Metrics 


To  test  for  effects  of  the  introduction  of  the  DoD/VA  diabetes 
guideline  on  service  utilization  and  prescription  patterns,  we  fit  a 
series  of  regression  models  to  predict  effects  on  each  of  the  six  mea¬ 
sures  for  diabetes  treatment.  We  present  in  this  appendix  tables  with 
descriptive  statistics  for  each  measure,  as  well  as  results  of  the  regres¬ 
sion  models  that  tested  for  possible  effects  of  guideline  implementa¬ 
tion  on  trends  for  each  measure. 

As  described  in  Chapter  Two,  the  study  sample  for  this  analysis 
consisted  of  all  members  of  the  larger  study  sample  who  were  enrolled 
for  the  entire  year  at  one  of  the  demonstration  MTFs  or  control 
MTFs.  Separate  samples  of  enrollees  were  defined  for  each  of  study 
years  one  and  two  based  on  enrollments  reported  in  the  encounter 
data  for  each  year. 

The  unit  of  analysis  for  all  of  the  measures  was  the  patient,  so 
there  was  one  data  record  for  each  patient  with  variables  for  each  of 
the  measures.  The  variables  for  primary  care  visits,  ER  visits,  and 
hospitalizations  were  continuous  variables  of  the  counts  of  relevant 
events.  The  variables  for  eye  examination  and  use  on  oral  hypoglyce¬ 
mic  agents  were  dichotomous  variables  that  indicated  the  presence  or 
absence  of  each  event. 

We  used  ordered  logistic  regression  models  to  test  the  size  and 
statistical  significance  of  effects  for  primary  care  visits  and  hospitaliza¬ 
tions  because  these  events  were  frequent  enough  to  yield  a  distribu¬ 
tion  of  patients  by  number  of  visits  or  inpatient  stays.  We  coded  a 
three-level  variable  for  primary  care  visits  (0,  1,  2,  3+  visits)  and  a 
two-level  variable  for  hospitalizations  (0,  1,  or  2+  stays).  Because  ER 
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visits  were  more  rare  events,  we  coded  a  dichotomous  variable  for 
each  parient  for  having  at  least  one  ER  visit  or  not.  We  used  standard 
logistic  regression  models  to  test  effects  on  ER  visits  as  well  as  effects 
on  eye  examinations  and  use  of  oral  hypoglycemic  agents. 

The  predictor  variables  in  the  models  included  dummy  variables 
for  each  quarter  with  quarter  four  omitted  as  the  referent  variable  (the 
last  quarter  before  the  demonstration  started) ,  a  dummy  variable  for 
demonstration  site,  and  variables  to  control  for  patient  characteristics. 
We  controlled  for  the  patient  characteristics  of  insulin  user,  gender, 
and  age  categories.  The  referent  age  category  in  our  models  was  46- 
64  years  and  the  other  categories  were  birth  to  17  years,  18  to  45 
years,  and  65  years  or  older. 

Guideline  effects  were  measured  using  interaction  terms  of 
demonstration  site  by  each  of  the  four-quarter  dummy  variables  for 
the  demonstration  period  (the  fifth  through  eighth  quarters).  The 
coefficient  on  each  quarter  variable  estimated  the  difference  in  a 
measure  between  demonstration  and  control  sites  relative  to  the  base¬ 
line  period — i.e.,  the  effects  of  the  demonstration. 

In  logistic  regression  models,  the  magnitude  of  effect  for  a  unit 
change  in  a  variable  can  expressed  as  an  odds  ratio,  which  is  obtained 
by  exponentiating  the  variable’s  coefficient.  An  odds  ratio  is  defined 
as  the  odds  that  an  outcome  variable  will  occur  divided  by  the  odds 
that  it  will  not  occur.  An  odds  ratio  for  a  predictor  variable  that  is 
equal  to  one  (equal  odds)  indicates  that  the  variable  has  no  effect  on 
the  occurrence  of  the  outcome.  An  odds  ratio  greater  than  one  indi¬ 
cates  that  the  variable  increases  the  probability  of  the  outcome  occur¬ 
ring,  and  an  odds  ratio  of  less  than  one  indicates  that  it  decreases  the 
probability.  We  report  here  both  the  statistical  significance  of  the 
predictor  variable  coefficients  and  the  odds  ratios  for  the  models 
estimated  for  the  six  diabetes  metrics. 

Primary  Care  Visits 

The  rates  of  primary  care  visits  by  group  and  individual  MTE  are 
reported  for  all  diabetic  patients  in  Table  B.l  and  for  noninsulin 
patients  in  Table  B.2.  Results  of  the  logistic  regression  analysis  for 
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Table  B.1 

Average  Annualized  Primary  Care  Visit  Rates  per  100  Patients  for  All 
Diabetic  Patients,  by  MTF  and  Quarter 


Facilities 

Study  Year  One  Quarters 

Study  Year  Two  Quarters 

1st 

2nd 

3rd 

4th 

1st 

2nd 

3rd 

4th 

Demonstration  Sites 

408 

396 

408 

432 

452 

412 

392 

404 

Demo  1 

316 

288 

284 

336 

352 

336 

308 

328 

Demo  2 

604 

604 

432 

528 

436 

352 

624 

544 

Demo  3 

456 

476 

444 

464 

452 

440 

412 

404 

Demo  4 

384 

380 

480 

428 

428 

344 

236 

284 

Demo  5 

492 

444 

560 

588 

692 

600 

636 

668 

Control  Sites 

320 

268 

240 

296 

304 

304 

304 

360 

Control  1 

352 

336 

320 

332 

324 

320 

296 

320 

Control  2 

128 

184 

248 

204 

396 

372 

280 

460 

Control  3 

276 

196 

272 

364 

304 

196 

284 

416 

Control  4 

360 

156 

20 

212 

232 

344 

372 

456 

Control  5 

288 

280 

208 

260 

300 

252 

244 

292 

NOTE:  Annualized  primary  care  visit 

rates 

are  calculated 

as  four  times  the 

number  of  visits  in  a 

quarter  divided  by  the  number  of  patients  in 

the  relevant 

study  year  and  applying  100  as  an  adjustment  factor  to  standardize  to  a 

1  rate 

per  100  patients. 

Table  B.2 

Average  Annualized  Primary  Care  Visit  Rates  per  100  Patients  for 
Non-Insulin  User  Diabetic  Patients,  by  MTF  and  Quarter 

Facilities 

Study  Year  Qne  Quarters 

Study  Year  Two  Quarters 

1st 

2nd 

3rd 

4th 

1st 

2nd 

3rd 

4th 

Demonstration  Sites 

380 

376 

388 

412 

424 

388 

372 

388 

Demo  1 

260 

256 

252 

300 

308 

296 

284 

304 

Demo  2 

572 

584 

392 

472 

444 

344 

568 

500 

Demo  3 

440 

460 

432 

452 

436 

428 

392 

396 

Demo  4 

360 

368 

444 

412 

412 

332 

232 

268 

Demo  5 

468 

412 

532 

560 

652 

560 

612 

636 

Control  Sites 

304 

256 

228 

288 

288 

292 

300 

348 

Control  1 

340 

324 

312 

328 

312 

312 

292 

316 

Control  2 

120 

180 

228 

184 

364 

360 

280 

460 

Control  3 

244 

168 

256 

356 

288 

168 

268 

348 

Control  4 

360 

148 

16 

208 

216 

332 

372 

444 

Control  5 

248 

264 

188 

240 

292 

240 

232 

288 

NOTE:  Annualized  primary  care  visit  rates  are  calculated  as  four  times  the 
number  of  visits  in  a  quarter  divided  by  the  number  of  patients  in  the  relevant 
study  year  and  applying  100  as  an  adjustment  factor  to  standardize  to  a  rate 
per  100  patients. 
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trends  in  primary  care  visits  for  all  diabetic  patients  in  our  sample  are 
reported  in  Table  B.2. 

The  lower  primary  care  visit  rates  for  the  demonstration  sites  in 
the  last  two  quarters  of  the  demonstration  period  are  statistically  sig¬ 
nificant  and  are  shown  in  the  coefficients  and  odds  ratios  for  the 
demo  *  quarter  interaction  terms  in  the  model  (see  Table  B.3).  The 
significant  odds  ratios  of  0.87  and  0.83  for  the  interaction  terms  indi¬ 
cate  that  patients  in  the  demonstration  sites  were  less  likely  to  use 
primary  care  visits  in  the  last  two  quarters.  The  nonsignificant  odds 
ratios  for  the  main  effect  variables  for  quarter  seven  and  quarter  eight 
represent  the  trend  for  the  control  sites. 


Table  B.3 

Ordered  Logistic  Regression  Model  of  Estimated  Guideline 
Effects  on  Number  of  Primary  Care  Visits  per  Diabetic 
Patient 


Parameter 

Estimated 

Coefficient 

Standard 

Error 

Odds 

Ratio 

Demonstration  (1,0) 

0  31  *** 

0.02 

1.36 

Quarter  1 

-0.01 

0.03 

1.00 

Quarter  2 

-0.10** 

0.03 

0.91 

Quarter  3 

-0.11** 

0.03 

0.89 

Quarter  5 

0.02 

0.05 

1.02 

Quarter  6 

-0.01 

0.05 

0.99 

Quarter  7 

0.03 

0.05 

1.03 

Quarter  8 

0.12** 

0.05 

1.13 

Male 

0.01 

0.01 

1.01 

Age  birth  to  17  years 

-0.01 

0.05 

1.00 

Age  18  to  45  years 

-0.42*** 

0.02 

0.66 

Age  65+  years 

-0.06** 

0.02 

0.94 

Insulin  user 

Q  52*** 

0.02 

1.68 

Interaction  terms 

Demo  *  quarter  5 

0.01 

0.05 

1.01 

Demo  *  quarter  6 

-0.03 

0.05 

0.97 

Demo  *  quarter  7 

-0.14** 

0.05 

0.87 

Demo  *  quarter  8 

-0.19** 

0.05 

0.83 

Intercept  1 

—2  24*** 

0.03 

Intercept  2 

—1  25*** 

0.03 

Intercept  3 

0.18 

0.03 

**  p  <  0.01,  ***  p  <  0.001. 

NOTE:  The  omitted  group  for  the  model  is  quarter  four,  which  is 
the  baseline  period  that  immediately  preceded  the  start  of 
implementation  activities  by  the  demonstration  MTFs. 
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Also  of  interest,  use  of  insulin  and  age  were  significant  predic¬ 
tors  of  use  of  primary  care  visits,  but  gender  was  not.  Insulin  users 
were  more  likely  to  use  primary  care  services.  Patients  in  referent  age 
group  of  46  to  64  years  were  the  heaviest  users  of  primary  care  ser¬ 
vices.  Compared  to  this  group,  younger  patients  used  much  less  pri¬ 
mary  care,  and  older  patients  used  slightly  less  care.  Similar  results 
were  obtained  for  regressions  modeled  separately  for  insulin  users  and 
nonusers  (not  presented  here). 

Use  of  Oral  Hypoglycemic  Agents  to  Control  Diabetes 

The  percentages  of  noninsulin  patients  prescribed  other  medications 
to  control  diabetes,  by  group  and  individual  MTF,  are  reported  in 
Table  B.4.  The  results  of  the  logistic  regression  analysis  for  this  mea¬ 
sure  are  reported  in  Table  B.5. 

For  this  analysis,  we  estimated  the  number  of  these  patients  for 
each  of  the  two  study  years,  and  the  regression  analysis  compared  the 
percentage  of  patients  who  used  other  medications  before  and  during 
the  demonstration  period.  The  combined  coefficients  and  odds  ratios 


Table  B.4 

Percentage  of  Non-Insulin  Dependent  Diabetic  Patients 
Prescribed  Oral  Hypoglycemic  Agents,  by 
Demonstration  and  Control  Sites  and  Year 


Facilities 

Percentage  of  Noninsulin  Patients 
on  Medications 

Study  Year  One 

Study  Year  Two 

Demonstration  Sites 

55.6 

58.4 

Demo  1 

56.8 

60.1 

Demo  2 

58.5 

63.1 

Demo  3 

57.9 

61.2 

Demo  4 

49.5 

49.2 

Demo  5 

51.5 

55.5 

Control  Sites 

48.1 

57.8 

Control  1 

55.1 

61.2 

Control  2 

12.7 

31.3 

Control  3 

44.1 

49.7 

Control  4 

46.3 

59.9 

Control  5 

45.7 

55.7 
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Table  B.5 

Logistic  Regression  Model  of  Estimated  Guideline  Effects 
on  Number  of  Non-Insulin  Dependent  Diabetic  Patients 
with  Other  Medication 


Parameter 

Estimated 

Coefficient 

Standard 

Error 

Odds 

Ratio 

Demonstration  (1,0) 

0.25*** 

0.05 

1.28 

Year  two 

0.39*** 

0.06 

1.48 

Male 

0.21*** 

0.03 

1.24 

Age  birth  to  17  years 

-3.45*** 

0.26 

0.03 

Age  18  to  45  years 

_0  QQ*** 

0.05 

0.41 

Age  65+  years 
Interaction  terms 

-0.24*** 

0.04 

0.79 

Demo  *  year  two 

-0.27** 

0.07 

0.76 

Intercept 

0.10* 

0.05 

*  p  <  0.05,  **  p  <  0.01,  ***  p  <  0.001. 

NOTE:  The  omitted  group  for  the  model  is  quarter  four,  which  is 
the  baseline  period  that  immediately  preceded  the  start  of 
implementation  activities  by  the  demonstration  MTFs. 


for  the  demonstration  and  control  sites  in  years  one  and  two  are  cal¬ 
culated  in  Table  B.6. 

In  the  baseline  year,  the  demonstration  sites  had  larger  percent¬ 
ages  than  the  control  sites  of  non-insulin  dependent  diabetic  patients 
using  oral  hypoglycemic  agents.  The  percentages  increased  in  the 
demonstration  year  for  both  the  demonstration  and  control  sites,  and 
both  increases  were  statistically  significant.  However,  the  increase  for 
the  demonstration  sites  was  smaller  than  that  for  the  control  sites, 
indicating  that  introduction  of  the  practice  guideline  did  not  have  an 
effect  on  these  trends. 

The  combined  effects  of  the  demonstration  and  year,  and  their 
interaction  term,  are  shown  in  Table  B.6  in  the  very  similar  odds 
ratios  for  the  control  and  demonstration  sites  for  year  two.  In  addi¬ 
tion,  use  of  insulin,  gender,  and  age  all  were  significant  predictors  of 
use  of  other  glycemic  control  medications.  Insulin  users  and  males 
were  more  likely  to  be  prescribed  these  medications.  Patients  in  refer¬ 
ent  age  group  of  46  to  64  years  were  the  heaviest  users  of  other  con¬ 
trol  medications  of  all  the  age  groups. 
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Annual  Eye  Examinations 

The  percentages  of  diabetic  patients  who  had  annual  eye  examina¬ 
tions,  by  group  and  individual  MTF,  are  reported  in  Table  B.7.  The 
results  of  the  logistic  regression  analysis  of  the  percentage  of  patients 
with  annual  eye  examinations  are  reported  in  Table  B.8. 

Overall,  the  demonstration  sites  had  larger  percentages  of  dia¬ 
betic  patients  with  annual  eye  examinations  than  the  control  sites  did. 


Table  B.6 

Comparison  of  Effect  of  Demonstration  on  Regression  Coefficients 
and  Odds  Ratios  for  Use  of  Other  Medications  to  Control  Glycemia 
by  Noninsulin  Patients 


Year  One 

Year  Two 

Odds 

Odds 

Coefficient 

Ratio 

Coefficient 

Ratio 

Control 

Year  two  variable 
Demonstration 

0.39 

1.48 

Demo  variable 

0.25 

1.28 

0.25 

Year  two  variable 

0.39 

Demo  *  year  two  interaction 

-0.27 

Sum  of  coefficients 

0.25 

1.28 

0.37 

1.45 

Table  B.7 

Percentage  of  Diabetic  Patients  with  a  Diabetes-Related 
Annual  Eye  Examination,  by  MTF  and  Year 


Facilities 

Percentage  of  Noninsulin  Patients 
with  Eye  Exam 

Study  Year  One 

Study  Year  Two 

Demonstration  Sites 

10.6 

19.5 

Demo  1 

4.5 

5.8 

Demo  2 

20.7 

25.8 

Demo  3 

14.6 

33.0 

Demo  4 

9.0 

0.3 

Demo  5 

15.9 

36.8 

Control  Sites 

5.2 

6.5 

Control  1 

0.3 

0.0 

Control  2 

6.9 

6.3 

Control  3 

18.8 

21.0 

Control  4 

12.1 

12.0 

Control  5 

0.5 

8.7 
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Table  B.8 

Logistic  Regression  Model  of  Estimated  Guideline  Effects 
on  Number  of  Diabetic  Patients  with  Annual  Eye 
Examinations 


Parameter 

Estimated 

Coefficient 

Standard 

Error 

Odds 

Ratio 

Demonstration  (1,0) 

0.78*** 

0.10 

2.18 

Year  two 

0.25* 

0.12 

1.28 

Male 

0.12** 

0.05 

1.13 

Age  birth  to  17  years 

-0.59** 

0.18 

0.55 

Age  18  to  45  years 

-0.36*** 

0.08 

0.70 

Age  65+  years 

-0.14** 

0.05 

0.87 

Interaction  terms 

0.08 

0.07 

1.08 

Demo  *  year  two 

0.47** 

0.13 

1.59 

Intercept 

-2.88*** 

0.09 

*  p  <  0.05,  **  p  <  0.01,  ***  p  <  0.001. 

NOTE:  The  omitted  group  for  the  model  is  year  one,  which  is  the 
baseline  period  that  immediately  preceded  the  start  of 
implementation  activities  by  the  demonstration  MTFs. 


as  shown  by  the  significant  coefficient  and  large  odds  ratio  (2.18)  for 
the  “demo”  variable.  The  percentage  with  eye  examinations  increased 
in  the  second  study  year  for  both  the  demonstration  and  control  sites 
(positive  and  statistically  significant  coefficient  for  the  “year  two” 
variable),  and  the  increase  was  greater  for  the  demonstration  sites 
(positive  and  statistically  significant  interaction  term  for  demo*year) . 
In  addition,  gender  and  age  were  significant  predictors  of  use  of  other 
control  medications,  but  insulin  use  was  not.  Males  were  more  likely 
than  females  to  be  prescribed  these  medications.  Patients  in  referent 
age  group  of  46  to  64  years  were  the  heaviest  users  of  primary  care 
services  of  all  the  age  groups. 

ER  Visits 

The  average  rates  of  ER  use  per  1,000  diabetic  patients,  by  group  and 
individual  MTF,  are  reported  in  Table  B.9.  The  results  of  the  ordered 
logistic  regression  analysis  of  emergency  department  visits  for  diabetes 
patients  are  presented  in  Table  B.IO  and  B.l  1. 

The  regression  results  in  Table  B.IO  include  all  the  demonstra¬ 
tion  sites,  and  the  results  in  Table  B.ll  exclude  one  demonstration 
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Table  B.9 

Average  Annualized  ER  Visit  Rates  per  100  Diabetic  Patients,  by 
Demonstration  and  Control  Sites  and  Quarter 


Study  Year  One  Quarters  Study  Year  Two  Quarters 


Facilities 

1st 

2nd 

3rd 

4th 

1st 

2nd 

3rd 

4th 

Demonstration  Sites 
Demo  sites  without 

4.4 

3.6 

4.0 

3.6 

4.0 

5.6 

8.0 

10.8 

Demo  y 

6.0 

3.2 

3.2 

2.4 

2.8 

2.0 

1.6 

3.2 

Demo  1 

6.4 

5.6 

5.2 

3.6 

4.8 

4.4 

3.6 

6.4 

Demo  2 

3.6 

3.6 

10.8 

14.4 

3.2 

6.4 

15.6 

0.0 

Demo  3 

0.8 

0.4 

1.2 

0.4 

0.4 

4.0 

14.4 

21.6 

Demo  4 

8.8 

6.0 

6.4 

7.2 

8.8 

8.8 

8.4 

4.8 

Demo  5 

4.4 

4.4 

5.2 

7.2 

6.0 

8.4 

3.6 

4.4 

Control  Sites 

2.0 

3.2 

3.2 

2.4 

2.8 

2.0 

1.6 

3.2 

Control  1 

1.6 

4.0 

3.2 

0.8 

1.6 

0.8 

1.6 

1.6 

Control  2 

2.4 

7.6 

2.4 

7.6 

25.2 

12.4 

12.4 

1.6 

Control  3 

3.6 

2.4 

8.0 

5.6 

4.4 

3.2 

2.0 

7.6 

Control  4 

0.0 

0.0 

0.0 

0.8 

0.0 

0.0 

0.0 

0.0 

Control  5 

4.8 

4.0 

4.8 

4.8 

4.4 

4.4 

1.2 

0.0 

^Separate  analyses  were  performed  with  and  without  Demo  3,  which  had  an 
unusual  trend  in  ER  visits  that  differed  from  the  other  sites. 


NOTE:  Annualized  primary  care  visit  rates  are  calculated  as  four  times  the 
number  of  visits  in  a  quarter  divided  by  the  number  of  patients  in  the  relevant 
study  year  and  applying  100  as  an  adjustment  factor  to  standardize  to  a  rate 
per  100  patients. 


site  that  was  an  outlier  with  a  large  increase  in  ER  visit  rates  during 
the  demonstration  period.  The  regression  results  for  all  demonstra¬ 
tion  sites  (Table  B.IO)  reflect  the  increased  rates  for  that  facility,  with 
increasingly  positive  and  statistically  significant  coefficients  on  the 
interaction  terms  for  demonstration  sites  hy  quarters  5  through  8. 
With  that  facility  omitted,  we  found  no  significant  trend  in  ER  visits 
for  the  demonstration  sites,  relative  to  rates  for  the  control  sites 
(Table  B.ll).  Thus,  these  results  indicate  that  the  demonstration  did 
not  contribute  to  decreased  use  of  the  ER  for  diabetes-related  prob¬ 
lems,  which  is  what  we  had  hypothesized  would  occur  as  more 
aggressive  care  management  practices  were  implemented. 

Also  of  interest,  use  of  insulin  and  age  were  significant  predic¬ 
tors  of  use  of  ER  visits,  but  gender  was  not.  Insulin  users  were  more 
likely  to  use  ER  services.  Pediatric  patients  (age  birth  to  17  years)  had 
the  highest  rates  of  ER  visits,  and  younger  adults  (age  1 8  to  45  years) 
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Table  B.10 

Logistic  Regression  Model  of  Estimated  Guideline  Effects 
on  Number  of  ER  Visits  per  Diabetic  Patient,  All 
Demonstration  and  Control  Sites 


Parameter 

Estimated 

Coefficient 

standard 

Error 

Odds 

Ratio 

Demonstration  (1,0) 

0.26 

0.14 

1.30 

Quarter  1 

0.14 

0.17 

1.15 

Quarter  2 

0.11 

0.17 

1.11 

Quarter  3 

0.21 

0.17 

1.24 

Quarter  5 

0.02 

0.29 

1.02 

Quarter  6 

-0.30 

0.32 

0.74 

Quarter  7 

-0.37 

0.33 

0.69 

Quarter  8 

-0.07 

0.30 

0.93 

Male 

0.06 

0.07 

1.07 

Age  birth  to  17  years 

1.26*** 

0.14 

3.51 

Age  18  to  45  years 

0.61*** 

0.1 

1.85 

Age  65+  years 

-0.01 

0.09 

0.99 

Insulin  user 

0  99*** 

0.08 

2.71 

Interaction  terms 

Demo  *  quarter  5 

0.12 

0.30 

1.12 

Demo  *  quarter  6 

0.72* 

0.33 

2.06 

Demo  *  quarter  7 

1.21** 

0.33 

3.35 

Demo  *  quarter  8 

ig*** 

0.30 

3.29 

Intercept  1 

—5  34*** 

0.18 

*  p  <  0.05,  **  p  <  0.01,  ***  p  <  0.001. 

NOTE:  The  omitted  group  for  the  model  is  quarter  four,  which  is 
the  baseline  period  that  immediately  preceded  the  start  of 
implementation  activities  by  the  demonstration  MTFs. 


also  had  higher  rates  than  patients  in  the  referent  age  group  (age  46  to 
64  years).  The  ER  visit  rate  for  older  patients  (age  65+  years)  did  not 
differ  significantly  from  that  of  the  referent  age  group. 

Frequency  of  Hospitalization 

The  average  numbers  of  hospitalizations  per  diabetic  patient,  by 
group  and  individual  MTF,  are  reported  in  Table  B.12  (all  hospitali¬ 
zations)  and  Table  B.13  (hospitalizations  for  diabetes).  The  results  of 
the  logistic  regression  analysis  of  trends  in  frequency  of  hospitaliza¬ 
tions  are  reported  in  Table  B.14. 

We  had  hypothesized  that  implementation  of  more  aggressive  care 
management  practices  by  the  demonstration  sites  would  lead  to 
reductions  in  hospitalization  rates  relative  to  those  for  the  control 
sites.  We  found  no  significant  trend  in  hospitalization  rates  for  the 
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demonstration  sites,  relative  to  rates  for  the  control  sites.  Thus,  these 
results  indicate  that  the  demonstration  did  not  contribute  to  a 
decrease  in  complications  or  other  health  problems  requiring  hospital 
inpatient  care  for  diabetic  patients. 


Table  B.11 

Logistic  Regression  Model  of  Estimated  Guideline  Effects 
on  Number  of  ER  Visits  per  Diabetic  Patient,  Excluding 
One  Demonstration  Site 


Parameter 

Estimated 

Coefficient 

Standard 

Error 

Odds 

Ratio 

Demonstration  (1,0) 

0.62*** 

0.14 

1.85 

Quarter  1 

0.14 

0.17 

1.15 

Quarter  2 

0.15 

0.17 

1.17 

Quarter  3 

0.20 

0.17 

1.22 

Quarter  5 

0.03 

0.29 

1.03 

Quarter  6 

-0.3 

0.32 

0.74 

Quarter  7 

-0.36 

0.33 

0.70 

Quarter  8 

-0.07 

0.30 

0.94 

Male 

0.05 

0.09 

1.05 

Age  birth  to  17  years 

1  ^Q*** 

0.15 

3.69 

Age  18  to  45  years 

0.68*** 

0.11 

1.95 

Age  65+  years 

-0.08 

0.12 

0.92 

Insulin  user 

Interaction  terms 

Q  gy*** 

0.09 

2.65 

Demo  *  quarter  5 

0.09 

0.30 

1.09 

Demo  *  quarter  6 

0.51 

0.34 

1.66 

Demo  *  quarter  7 

0.26 

0.35 

1.30 

Demo  *  quarter  8 

0.11 

0.32 

1.11 

Intercept  1 

—5  35*** 

0.17 

***  p  <  0.001. 

NOTE:  The  omitted  group  for  the  model  is  quarter  four,  which  is 
the  baseline  period  that  immediately  preceded  the  start  of 
implementation  activities  by  the  demonstration  MTFs. 


Table  B.12 

Average  Number  of  Total  Hospitalization  per  100  Diabetic  Patients,  by 
Demonstration  and  Control  Sites  and  Quarter 


Average  Number  of  Total  Hospitalizations  per  100 
Patients 


Study  Year  One  Study  Year  Two 


Facilities 

1st 

2nd 

3rd 

4th 

1st 

2nd 

3rd 

4th 

Demonstration  Sites 

42.0 

36.8 

38.8 

41.2 

45.2 

42.0 

39.2 

41.6 

Demo  1 

40.0 

39.6 

34.0 

38.4 

42.8 

40.4 

38.4 

36.8 

Demo  2 

21.6 

7.2 

10.8 

14.4 

43.6 

50.0 

65.6 

31.2 
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Table  B.12 — continued 


Average  Number  of  Total  Hospitalizations  per  100 
Patients 


Study  Year  One  Study  Year  Two 


Facilities 

1st 

2nd 

3rd 

4th 

1st 

2nd 

3rd 

4th 

Demo  3 

57.6 

42.8 

47.6 

45.2 

55.6 

44.8 

42.4 

51.6 

Demo  4 

31.6 

22.4 

32.4 

38.4 

28.8 

41.6 

32.8 

34.0 

Demo  5 

20.8 

30.0 

36.8 

46.0 

42.0 

39.6 

35.2 

38.4 

Control  Sites 

27.6 

29.6 

28.8 

27.2 

26.8 

26.4 

24.0 

29.2 

Control  1 

34.0 

31.2 

27.6 

26.4 

22.8 

27.2 

25.2 

26.4 

Control  2 

17.6 

20.0 

22.8 

25.2 

16.8 

25.2 

0.0 

62.4 

Control  3 

29.6 

35.2 

35.2 

35.2 

41.6 

34.8 

20.8 

34.0 

Control  4 

13.2 

20.4 

22.8 

21.2 

24.4 

18.4 

14.4 

20.8 

Control  5 

26.8 

34.4 

36.4 

29.6 

34.8 

28.0 

44.8 

41.2 

NOTE:  Annualized 

primary  care  visit 

rates 

are  calculated 

as  four 

times  the 

number  of  visits  in  a  quarter  divided  by  the  number  of  patients  in  the  relevant 
study  year  and  applying  100  as  an  adjustment  factor  to  standardize  to  a  rate  per 
100  patients. 


Table  B.13 

Average  Number  of  Diabetes-Related  Hospitalizations  per  100  Diabetic 
Patients,  by  Demonstration  and  Control  Sites  and  Quarter 


Average  Number  of  Diabetes-Related  Hospitalizations 
per  100  Patients 


Study  Year  One  Study  Year  Two 


Facilities 

1st 

2nd 

3rd 

4th 

1st 

2nd 

3rd 

4th 

Demonstration  Sites 

30.0 

28.8 

26.0 

29.2 

34.0 

32.0 

29.6 

32.4 

Demo  1 

31.6 

32.4 

26.8 

31.6 

34.0 

32.8 

29.2 

30.8 

Demo  2 

18.0 

7.2 

7.2 

14.4 

28.0 

34.4 

50.0 

22.0 

Demo  3 

38.4 

32.0 

28.4 

28.4 

2.4 

32.4 

32.8 

41.2 

Demo  4 

20.0 

16.4 

18.4 

25.2 

18.4 

29.6 

22.8 

25.6 

Demo  5 

14.4 

25.2 

26.8 

29.2 

30.0 

30.4 

26.0 

23.6 

Control  Sites 

20.0 

19.6 

21.2 

18.4 

19.2 

16.8 

16.8 

22.4 

Control  1 

25.2 

21.6 

22.8 

17.2 

18.0 

18.0 

20.0 

22.4 

Control  2 

2.4 

0.0 

2.4 

5.2 

4.0 

8.4 

0.0 

16.8 

Control  3 

21.6 

18.4 

26.4 

24.0 

22.0 

24.0 

14.0 

26.0 

Control  4 

8.0 

15.6 

14.8 

16.4 

18.8 

9.6 

7.6 

12.0 

Control  5 

22.0 

26.0 

26.8 

24.0 

26.8 

20.0 

28.0 

35.6 

NOTE:  Annualized 

primary  care  visit 

rates 

are  calculated 

as  four 

times  the 

number  of  visits  in  a  quarter  divided  by  the  number  of  patients  in  the  relevant 
study  year  and  applying  100  as  an  adjustment  factor  to  standardize  to  a  rate  per 
100  patients. 
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Table  B.14 

Ordered  Logistic  Regression  Model  of  Estimated  Guideline 
Effects  on  Number  of  Hospitalization  per  Diabetic  Patient 


Parameter 

Estimated 

Coefficient 

Standard 

Error 

Odds 

Ratio 

Demonstration  (1,0) 

0.13 

0.10 

1.14 

Quarter  1 

0.05 

0.12 

1.05 

Quarter  2 

0.16 

0.12 

1.17 

Quarter  3 

-0.1 

0.12 

0.90 

Quarter  5 

0.14 

0.20 

1.15 

Quarter  6 

-0.06 

0.20 

0.94 

Quarter  7 

0.26 

0.21 

1.29 

Quarter  8 

0.37 

0.19 

1.45 

Male 

0.17** 

0.06 

1.19 

Age  birth  to  17  years 

—0  7 1  *  *  * 

0.18 

0.49 

Age  18  to  45  years 

—0  93*** 

0.10 

0.40 

Age  65+  years 

0.01 

0.07 

1.01 

Insulin  user 

1  Qg*** 

0.08 

2.96 

Interaction  terms 

Demo  *  quarter  5 

0.03 

0.21 

1.03 

Demo  *  quarter  6 

0.17 

0.21 

1.19 

Demo  *  quarter  7 

-0.19 

0.22 

0.83 

Demo  *  quarter  8 

-0.2 

0.20 

0.82 

Intercept  1 

—2  05*** 

0.12 

Intercept  2 

0.93 

0.12 

**  p  <  0.01,  ***  p  <  0.001. 

NOTE:  The  omitted  group  for  the  model  is  quarter  four,  which  is 
the  baseline  period  that  immediately  preceded  the  start  of  imple¬ 
mentation  activities  by  the  demonstration  MTFs. 


Use  of  insulin,  gender,  and  age  all  were  significant  predictors  of 
hospitalization  rates.  Insulin  users  and  male  diabetic  patients  were 
more  likely  to  be  hospitalized.  Pediatric  patients  (age  birth  to  17 
years)  and  younger  adults  (age  1 8  to  45  years)  had  lower  hospitaliza¬ 
tion  rates  than  patients  in  the  referent  age  group  (age  46  to  64  years) 
did,  but  the  rate  for  older  patients  (age  65+  years)  did  not  differ  sig¬ 
nificantly  from  that  of  the  referent  age  group. 


APPENDIX  C 

Graphic  Representation  of  Baseline  Service  Use 
Data 


Figure  Cl 

Outpatient  and  ER  Visits  to  MTF  and  Network  Providers  by  Patient 
Enrollment  Status  and  by  Study  Year 
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Figure  C.2 

Outpatient  and  ER  Visits  of  MTF  and  Network  Enrollees  and  Nonenrollees 
to  MTF  and  Network  Providers,  by  Provider  Type  and  Study  Year 
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Figure  C.3 

Enrollment  Status  of  Patients  for  Inpatient  Admissions  at  MTFs  and 
Network  Providers,  by  Study  Year 
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Figure  C.4 

Inpatient  Admissions  at  MTFs  and  Network  Providers,  by  Enrollment 
Status  of  Patients  and  Study  Year 


■  MTF  provider 
□  Network  provider 


Prime  to  Prime  to  Not  Prime  to  Prime  to  Not 

MTF  network  enrolled  MTF  network  enrolled 


RAND  MG277-C.4 


Study  year  one 


Study  year  two 


Bibliography 


Bero,  L.  A.,  R.  Grilli,  J.  M.  Grimshaw,  E.  Harvey,  A.  D.  Oxman,  M.  A. 
Thomson,  “Glosing  the  Gap  Between  Research  and  Practice:  An  Over¬ 
view  of  Systematic  Reviews  of  Interventions  to  Promote  the  Imple¬ 
mentation  of  Research  Findings,”  The  Gochrane  Effective  Practice  and 
Organization  of  Gate  Review  Group,  BMJ  {British  Medical  Journal),  Vol. 
317, 1998, pp.  465-468. 

Garter,  G.  M.,  and  G.  Melnick,  How  Services  and  Costs  Vary  by  Day  of  Stay 
for  Medicare  Hospital  Stays,  Santa  Monica,  Galif;  RAND  Gorporation, 
R-3870-ProPAG,  1990. 

Genters  for  Disease  Gontrol  and  Prevention  (GDG),  “National  Diabetes 
Fact  Sheet,”  published  on  the  GDG  Web  site,  www.cdc.gov/diabetes/ 
pubs/facts98.htm,  March  28,  2001. 

Ghodoff,  P.,  and  K.  Growley,  “Glinical  Practice  Guidelines:  Roadblocks  to 
Their  Acceptance  and  Implementation,”  Journal  of  Outcomes  Manage¬ 
ment,  Vol.  2,  1995,  pp.  5-10. 

Gox,  S.,  P.  Wilcock,  and  J.  Young,  “Improving  the  Repeat  Prescribing 
Process  in  a  Busy  General  Practice:  A  Study  Using  Gontinuous  Quality 
Improvement  Methodology,”  Quality  Health  Care,  Vol.  8,  No.  2,  June 
1999,  pp.  1 19-125. 

Eastwood,  A.  J.,  and  T.  A.  Sheldon,  “Organisation  of  Asthma  Gate:  What 
Difference  Does  It  Make?  A  Systematic  Review  of  the  Literature,” 
Quality  Health  Care,  Vol.  5,  1996,  pp.  134-143. 

Evans,  R.  S.,  S.  L.  Pestotnik,  D.  G.  Glassen,  et  ah,  “A  Gomputer-Assisted 
Management  Program  for  Antibiotics  and  Other  Antiinfective  Agents 


145 


146  Implementation  of  the  Diabetes  Practice  Guideline  in  AMEDD 


[See  Comments],”  New  England  Journal  of  Medicine,  Vol.  338,  1998,  pp. 
232-238. 

Farley,  Donna  O.,  Georges  Vernez,  Elaine  S.  Quiter,  and  Shan  Cretin, 
“Early  Lessons  from  the  Low  Back  Pain  Practice  Guideline  Demonstra¬ 
tion,”  Santa  Monica,  Calif;  RAND  Corporation,  No.  PM-1008-A, 
November  1999. 

Gandhi,  T.  K.,  A.  L.  Puopolo,  P.  Dasse,  J.  S.  Haas,  H.R.  Burstin,  E.  F. 
Cook,  and  T.  A.  Brennan,  “Obstacles  to  Collaborative  Quality 
Improvement:  The  Case  of  Ambulatory  General  Medical  Care,”  Interna¬ 
tional  Journal  of  Quality  Health  Care,  Vol.  12,  No.  2,  April  2000,  pp. 
115-123. 

Grilli,  R.,  and  J.  Lomas,  “Evaluating  the  Message:  The  Relationship 
Between  Compliance  Rate  and  the  Subject  of  a  Practice  Guideline,” 
Medical  Care,  Vol.  32,  1994,  pp.  202-213. 

Grimshaw,  J.  M.,  L.  Shirran,  R.  E.  Thomas,  G.  Mowatt,  C.  Fraser,  L.  Bero, 
et  ah,  “Changing  Provider  Behaviour;  An  Overview  of  Systematic 
Reviews  of  Interventions,”  Vol.  39,  No.  8,  Supplement  2,  Medical  Care, 
2001,  pp.  2-45,. 

Grol,  R.,  J.  Daihuijsen,  S.  Thomas,  C.  Veld,  G.  Rutten,  H.  Mokkink, 
“Attributes  of  Clinical  Guidelines  That  Influence  Use  of  Guidelines  in 
General  Practice:  Observational  Study,”  BMJ  (British  Medical  Journal), 
Vol.  317,  1998,  pp.  858-861. 

Gustafson,  D.  H.,  W.  L.  Cats-Baril,  and  F.  Alemi,  Systems  to  Support  Health 
Policy  Analysis:  Theory,  Models,  and  Uses,  Chicago:  Health  Administra¬ 
tion  Press,  1992. 

Hargraves,  J.  L.,  R.  H.  Palmer,  E.  J.  Orav,  and  E.  A.  Wright,  “Practice 
Characteristics  and  Performance  of  Primary  Care  Practitioners,”  Medical 
Care,  Vol.  34,  1996,  pp.  SS67-SS76. 

Keller,  G.  A.,  “Management  for  Quality:  Continuous  Quality  Improvement 
to  Increase  Access  to  Outpatient  Mental  Health  Services,”  Psychiatric 
Services,  Vol.  48,  No.  6,  June  1997,  pp.  821-825. 

Lescoe-Long,  M.,  and  M.  J.  Long,  “Defining  the  Utility  of  Clinically 
Acceptable  Variations  in  Evidence-Based  Practice  Guidelines  for  Evalua¬ 
tion  of  Quality  Improvement  Activities,”  Evaluation  and  the  Health  Pro¬ 
fessions,  Vol.  22,  No.  3,  September  1999,  pp.  298-324. 


Bibliography  147 


Lewis,  S.,  “Paradox,  Process,  and  Perception;  The  Role  of  Organizations  in 
Clinical  Practice  Guidelines  Development,”  CMAJ  {Canadian  Medical 
Association  Journal), \o\.  153,  1995,  pp.  1073-1077. 

Motwani,  J.,  D.  Klein,  and  S.  Navitskas,  “Striving  Toward  Continuous 
Quality  Improvement:  A  Case  Study  of  Saint  Mary's  Hospital,”  Health 
Care  Management  Review,  Vol.  18,  No.  2,  December  1999,  pp.  33-40. 

Nicholas,  W.,  D.  Farley,  M.  Vaiana,  and  S.  Cretin,  “Putting  Practice 
Guidelines  to  Work  in  the  Army  Medical  Department:  A  Guide  for 
Action,”  Santa  Monica,  Calif;  RAND  Corporation,  PM-1023-A,  Janu¬ 
ary  2000. 

Palmer,  R.  H.,  J.  L.  Hargraves,  “Quality  Improvement  Among  Primary 
Care  Practitioners:  An  Overall  Appraisal  of  Results  of  the  Ambulatory 
Care  Medical  Audit  Demonstration  Project,”  Medical  Care,  Vol.  34, 
Supplement  9,  September  1996,  pp.  SS102-SS113. 

Sasala,  D.  B.,  and  D.  A.  Jasovsky,  “Using  a  Hospitalwide  Performance 
Improvement  Process  for  Patient  Education  Documentation,”  Joint 
Commission  Journal  on  Quality  Improvement,  Vol.  24,  No.  6,  June  1998, 
pp.  313-322. 

Savitz,  L.  A.,  A.  D.  Kaluzny,  “Assessing  the  Implementation  of  Clinical 
Process  Innovations:  A  Cross-Case  Comparison,”  Journal  of  Healthcare 
Management,  Vol.  45,  No.  6,  November-December  2000,  pp.  366-379; 
discussion  pp.  379-380. 

Senge,  P.  M.,  The  Fifth  Discipline:  The  Art  and  Practice  of  The  Learning 
Organization,  New  York;  Doubleday/Currency,  1990. 

Solberg,  L.  L,  L.  A.  Reger,  T.  L.  Pearson,  L.  M.  Cherney,  P.  J.  O’Connor, 
S.  L.  Freemen,  S.  L.  Lasch,  and  D.  B.  Bishop,  “Using  Continuous  Qual¬ 
ity  Improvement  to  Improve  Diabetes  Care  in  Populations:  The  IDEAL 
model  (Improving  care  for  Diabetics  through  Empowerment  Active 
collaboration  and  Leadership),”  Joint  Commission  Journal  on  Quality 
Improvement,  Vol.  23,  No.  11,  November  1997,  pp.  581-592. 

Shortell,  S.  M.,  C.  L.  Bennett,  and  G.  R.  Byck,  “Assessing  the  Impact  of 
Continuous  Quality  Improvement  on  Clinical  Practice:  What  It  Will 
Take  to  Accelerate  Progress  [see  comments],”  Milbank  Quarterly,  Vol. 
76,  1998,  pp.  593-624,  510. 

Vernez,  G.,  D.  Farley,  S.  Cretin,  W.  Nicholas,  K.  J.  Dolter,  M.  Lovell,  and 
J.  Schmith,  “Proposed  Managerial  Structure  to  Support  Army-Wide 


148  Implementation  of  the  Diabetes  Practice  Guideline  in  AMEDD 


Implementation  of  Clinical  Practice  Guidelines,”  Santa  Monica,  Calif: 
RAND  Corporation,  PM-1006-A,  June  2000. 

Von  Korff,  M.,  J.  Gruman,  J.  Schaefer,  S.  J.  Curry,  and  E.  H.  Wagner, 
“Collaborative  Management  of  Chronic  Illness,”  Annals  of  Internal  Medi¬ 
cine,  Vol.  127,  1997,  pp.  I097-II02. 

Wagner,  E.  H.,  B.  T.  Austin,  and  M.  VonKorff,  “Organizing  Care  for 
Patients  with  Chronic  Illness,”  Milbank  Quarterly,  Vol.  74,  1996,  pp. 
51 1-544. 


